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AUTHOR'S PREFACE 


This book is the author's doctoral dissertation, presented in January, 1947, before 
the academic council of the Molotov Energy Institute in Moscow. Despite the fact that 
many works devoted to noise immunity have appeared in the time that has elapsed since 
the writing of this dissertation, not all of the topics considered in it have as yet appeared 
in print. Considering the great interest shown in these matters, and also the number of 
references made to this work in the literature, the author has deemed it appropriate to 
publish it, without introducing any supplementary material. However, in preparing the 
manuscript for publication, it was somewhat condensed, at the expense of material of 
secondary interest. Moreover, Chapter 2, which contains auxiliary mathematical material, 
has been revised somewhat, to make it easier reading, and some of the material has been 


relegated to the appendices. 


The author 
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EXPLANATORY NOTE 


The study of probability theory and its applications has had a long and illustrious 
history in Russia, beginning in the earliest days of the 18th century, and continuing in an 
unbroken line down to the present generation. In our time we have seen a realization that 
in many aspects of science, technology, and human behavior the element of randomness is 
so fundamental that often one can hardly define a meaningful problem, much less solve it, 
without using probability theory. During the rapid technological developments of the 
World War II period, the communication and detection arts underwent such a realization, 
and as would be expected, statistical communication theory (or information theory) has 
occupied some of the best minds among mathematicians and engineers in the Soviet Union 
just as it has elsewhere. 

One of the most important Soviet contributions, and one that was until recently 
virtually unknown outside the U.S.S.R., was the 1947 doctoral dissertation of 
V. A. Kotel'nikov, at that time a 40-year-old communications engineer, who had already 
in his younger days (1933) become well known for his work on sampling theorems for 
band-limited functions. Kotel'nikov's dissertation constituted an extensive analysis of 
the effects of additive gaussian noise on communication systems, and of what could be 
done at the receiver to minimize them. Unlike Shannon's information theory, he did 
not go extensively into the implications of a freedom to choose complicated transmitter 
signals. 

Many Soviet contributions to the statistical communication art are fairly well known 
tous. Every student of these matters knows the names Khinchin and Kolmogorov as 
partners with Western mathematicians (notably Norbert Wiener) in the early development 
of spectral and filtering theories for random functions. Yet few of us have been aware 
that there existed in 1947 in this dissertation a statistical analysis of communication 
problems using what we now call decision theory techniques and anticipating by several 
years much of the work of Woodward, Davies, Siegert and others, with which we are 
more conversant. 

This book is a verbatim translation of "The Theory of Potential Noise Immunity", 
published by the State Power Engineering Press in 1956. As the Author's Preface has just 
indicated, it is essentially identical to the 1947 dissertation. In preparing this English 
edition, no technical editing has been done other than the correction of misprints. The 
present volume thus retains the exact flavor of the original, allowing one to see from 


hindsight which of Kotel'nikov's many highly original ideas have been developed further 


and which have not. 


By no means all of Kotel'nikov's results have since been obtained independently by 
others, and thus the volume should be of much more than just historical interest. 

Perhaps the reader will be aided by the following few comments which should make 
the unfamiliar terminology a little easier to follow, and should clarify the relationship 
between this and other works in the communication theory field. 

First of all there is the question of just what is meant by "noise immunity". As used 
here, it is a generic term with a different meaning for different situations. For Part II, 
the case in which communication takes place by transmitting one out of a finite number of 
possible signals the term refers to probability of no error. (Part II discusses what we 
would now call the 'multiple-alternative decision" problem.) Part III treats the situation 
in which a continuum of transmitted signals is assumed (a parameter x ranging over some 
interval taking the place of the previous discrete index of the possible signals); i.e. , the 
problem of "parameter estimation". Here a greater noise immunity refers to a decrease 
in mean square value of the error between the value of X indicated by the receiver and that 
actually transmitted. And then when the author treats in Part IV the case of signalling 
using waveforms (the parameter now being replaced by a function of time in some time 
interval) an increased noise immunity refers to a decrease in the average noise power that 
additively corrupts the receiver output. In other words, the author is dealing with the 
mean-square error between the modulating signal entering the transmitter, and that repro- 


" one of 


duced by the receiver, Note that the author never says that noise immunity "is 
these three things, but rather "is characterized by'' one of them. This usage persists in 
the large number of Soviet papers that have continued Kotel'nikov's work. 

The development presented here is notable in its absence of any dependence on an 
advanced mathematical background. The reader possessing a passing familiarity with 
Fourier series, discrete and continuous probabilities and probability densities (simple, 
joint, and conditional) and the notion of statistical independence will have no trouble, At 
several points some known results of probability theory are invoked without reference or 
proof, (One that the beginning reader might not be familiar with is the Central Limit 
Theorem, Equation 2-33.) However, these instances are rare; by and large the treatment 
is completely self-sufficient, 

Kotel'nikov made extensive use of geometric models of the signalling and detection 
processes as operations on vectors in multi-dimensional space, an artifice that Shannon 
introduced later. The reader will find these geometric interpretations very helpful. The 
material of each chapter is reviewed in terms of the geometric model at the end of the 


chapter, 


Paul E. Green, Jr. 
M.I.T. Lincoln Laboratory 
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PART I 
AUXILIARY MATERIAL 


CHAPTER 1 
INTRODUCTION 


1-1 Methods of combating noise 

Ordinarily, a radio receiver is acted upon not only by disturbances (signals) produced 
by the radio transmitter, but also by disturbances (noise) produced by a large variety of 
sources, The noise combines with the signals and corrupts them; in the case of telegraphic 
reception this leads to errors, and in the case of telephonic reception to background noise, 
Static, etc, When the signals are too small compared to the noise, reception becomes 
impossible. 


The following methods of combating noise are used: 


1. Decreasing the strength of the noise by taking action against their sources, 

2. Increasing the ratio of the strength of the signals to that of the noise by increasing 
the transmitter power and by using directional antennas, 

3. Improving the receivers, 

4, Changing the form of the signals while keeping their power fixed. (This is done 


with the aim of facilitating the combating of noise in the receiver. ) 


The first two methods are not considered in this book, which is devoted rather to the 
last two methods, and has as its goal to examine whether it is possible to decrease the 
effect of noise by improving the receivers, given the existing kinds of signals. In particular, 
what can be achieved in combating noise by changing the form of the signals? What form of 


signals is optimum for this purpose? 


1-2 Classification of noise 


We can classify the noise which impedes radio reception into the following categories: 


A. Sinusoidal noise consisting of one or a finite number (usually small) of sinusoidal 
oscillations. This category of noise includes interference from the parasitic radiation of 
one or more radio stations operating at frequencies near that of the station being received. 

B, Impulse noise consisting of separate impulses which follow one another at such 
large time intervals that the transients produced in the receiver by one impulse have sub- 
stantially died out by the time the next impulse arrives. This category of noise includes 


some kinds of atmospheric noise and noise from electrical apparatus. 


C. Normal fluctuation noise: or, as it is sometimes called, smoothed-out noise. 
This also consists of separate impulses, occurring at random time intervals, but the 
impulses follow one another so rapidly that the transients produced in the receiver by the 
individual impulses are superimposed in numbers large enough to warrant the application 
of the laws of large numbers of probability theory. This category of noise includes vacuum 
tube noise, noise due to the thermal motion of electrons in circuits, and some kinds of 
atmospheric noise and noise from electrical apparatus, At very high frequencies this 
kind of noise is encountered almost exclusively. 

D, Impulse noise of an intermediate type, which occurs when the transients produced 
in the receiver by the individual impulses are superimposed, but not in numbers large 
enough to warrant the application with sufficient accuracy of the laws of large numbers. 
This kind of noise is intermediate between categories Band C. 

Methods of studying the action of sinusoidal and impulse noise on radio receivers are 
at present quite well developed. The study of impulse noise of the intermediate type, 
when the transients produced by the individual impulses are just beginning to be super- 
imposed, is much more difficult. Moreover, in this case, we need to know not only the 
shapes of the separate impulses, but also the probability of superposition of impulses 
which have various shapes, and which obey various time distributions. In most cases we 
do not have this information about the noise, and it seems to be quite difficult to obtain. 
For these reasons, and also because noise of category C is often encountered, in what 
follows we shall consider only noise of this latter category; we shall often designate 


normal fluctuation noise simply as noise, 


1-3 Messages and signals 


By a message we shall mean that which is to be transmitted. The messages with 


which we shall be concerned can be divided into three categories, 


A, Discrete rnessages, 
B. Messages in the form of separate nurnbers (parameters), which can take on any 


values in certain ranges, 


C, Messages in the form of wave trains, which can assume a continuous infinity of 


different waveforms, 


The messages which are transmitted in telegraphy belong to the category of discrete 
messages. In this case, they consist of discrete letters, numerals, and characters, 
which can take on a finite number of discrete values. Moreover, in many instances, the 


messages transmitted in remote-control systems belong to this category. 


l. The use of the word "normal" alludes to the fact that we deal here with one of a 
variety of possible fluctuation processes, 


In the case of the transmission of individual measurements with the aid of telemeter- 
ing, the messages consist of the values of certain parameters (e. g., temperature, 
pressure, etc.) measured at given time intervals. These quantities usually take on 
arbitrary values lying within certain ranges. Thus, in this case we cannot restrict our- 
selves to a finite number of possible discrete messages. Messages of this kind belong to 
category B, 

In the case of telephony, the messages are acoustical vibrations, or the electrical 
vibrations taking place in the microphone, which can take on an infinite number of different 
forms. These messages belong to category C, In television, the oscillations acting on 
the transmitter can be taken as the message; this message also belongs to the last category. 

We shall assume that some variation in voltage, produced by the operation of the trans- 
mitter, acts upon the receiver input. We have called this variation in voltage a signal, 
Clearly, there will be a signal corresponding to each possible trarsmitted message. The 


receiver must use this voltage waveform (i.e. , signal) to reproduce the message to which 


the signal corresponds, 


1-4 The contents of this book 


In this book we consider the influence of normal fluctuation noise on the transmission 
of messages, The problem which will concern us is the following: We assume that when 
the noise perturbation is not superimposed on the signal, then the receiver will reproduce 
the transmitted message exactly. If noise is added to the signal, then the sum of two 
voltages will act upon the receiver input, i.e. , the signal voltage plus the noise voltage. 
In this case, depending on the sum voltage, the receiver will reproduce some message or 
other, which in a given instance may be different from the one that was transmitted. 
Clearly, each sum voltage which acts upon the receiver produces the particular message 
which corresponds to it, This correspondence may be different for different receivers, 
Depending on this correspondence, a receiver will be more or less subject to the influence 
of noise for a given kind of transmission. We shall find out what this correspondence 
ought to be for the message corruption to be the least possible, The receiver which has 
this optimum correspondence will be called ideal, 

Next we shall determine the message perturbation which results when noise is added 
to the signals, and when the reception is with an ideal receiver; the message perturbation 
obtained in this way will be the least possible under the given conditions, i.e., for real 
receivers under the same conditions, the message perturbation cannot be less. The noise 
immunity characterized by this least possible message perturbation will be called the 
optimum noise immunity. This noise immunity can be approached in real receivers if the 
receiver is close to being ideal, but it cannot be exceeded. By comparing the optimum 
noise immunity with the noise immunity afforded by real receivers, we can judge how 


close the latter are to perfection, and how much the noise immunity can be increased by 


improving them, i.e. , to what extent it is advisable to work on further increasing the 
noise immunity for a given means of communication. Knowledge of the optimum noise 
immunity makes it easy to discover and reject methods of communication for which this 
noise immunity is low compared with other methods. This can be done without reference 
to the method of reception, since real receivers cannot achieve noise immunity greater 
than the optimum, By comparing the optimum noise immunity for different means of 
communication, we can easily explain (as will be seen subsequently) the basic factors on 
which the immunity depends, and thereby increase the immunity by changing the means of 
communication. In the book, these matters are illustrated by a whole series of examples 
which have practical interest. However, the examples considered are far from exhausting 
all possible cases in which one can apply the methods of studying noise immunity developed 
here, 

In the book, all questions are discussed in connection with radio reception, in the 
interest of greater clarity; however, all that is said is directly applicable to other fields, 
like, for example, cable communication, acoustical and hydroacoustical signaling, etc. 
Moreover, in the book, all signal and noise disturbances are considered to be oscillations 
of voltage; however, nothing is changed if we consider instead oscillations of current, 
acoustical pressure, or of any other quantity which characterizes the disturbance acting on 
the receiver. 

This book does not consider certain irregular perturbations of the signals, which can 
strongly affect both the operation of radio receivers and their noise immunity. Examples 
of such perturbations are fading, echo phenomena, etc. Moreover, it should be kept in 
mind that in this book the word noise is henceforth (for brevity) understood to refer to 


normal fluctuation noise; indeed, this is the only kind of noise which will be considered. 


CHAPTER 2 
AUXILIARY MATHEMATICAL MATERIAL 


2-1 Some definitions 


We now introduce some definitions which simplify the subsequent analysis. We assume 
that all waveforms under consideration lie in the interval -T/2, + T/2, which is obviously 
always the case for sufficiently large T. 


The mean value of a waveform A(t) over the interval T is designated by 


A(t) dt (2-1) 


By the scalar product of two functions A(t) and B(t), we understand the mean value of 


their product over the interval -T/2,+T/2. Thus, the scalar product is 


+T/2 
AG) BI) = 5 if A(t) B(t) dt (2-2) 
-T/2 
It is clear from the definition that 
A(t) B(t) = Bt) A(t) (2-3) 
Furthermore 
A(t t) + C(t) ]= A(t) Blt) + Alt) C(t) (2-4) 
and 


faA(t)) [bB(t)]= ab A(t) Bit) (2-5) 
where a and b are arbitrary constants. Thus, the scalar product of functions has the 
same properties as the scalar product of vectors; instead of scalars we have constants, 
and instead of vectors we have functions. 


We write 


Av(t)= A(t) A(t) = = A(t) dt (2-6) 


In what follows, we shall often encounter the quantity 


+7T/2 
T A(t) = J A2(t) dt (2-7) 
Tf 2. 


This quantity will be called the specific energy of the waveform A(t), It equals the energy 


expended in a resistance of 1 ohm acted upon by the voltage A(t). The quantity 


Vo arity (2-8) 


will be called the effective value of the waveform A(t). A function with effective value 


unity is said to be normalized. 
If two functions differ only by a constant, they are said to coincide in direction. The 


normalized function which coincides in direction with a given function A(t) is obviously 


A(t 
eo NE Saee (2-9) 
Av (t) 
We shall say that the functions A(t), A,(t), eG AL (t) are (mutually) orthogonal, if 
a = - 
A(t) Apt) 0 (2-10) 


for all 1 <i, L< n, except when i= Jf. 


2-2 Representation of a function as a linear combination of orthonormal functions 


If the system of functions 


C y(t), Colt), «.. » C(t) (2-11) 
satisfies the equations 
2 
C(t) =l1 (2-12) 
oma) Colt) = 0 (2-13) 


where 1 <k, L< n and k #£ ZL, we say that it is a system of orthonormal functions. An 
example of such a system of functions is the system 
T(t) = 1, 
= i 2n 
T(t) Vz sin zt, 


L(t) =V¥2 cos a 


2 : an 
I, (t) =f2 sin 2 Tt 


t, 


(2-14) 
I(t) =V2 cos 2 a te, 
Ime} t) =V2 sinm an t, 
Lo walt) =V¥2 cos m oie t, 
since for this system the relations 
rea a ere 
cit) = 1, Tit) I(t) =0 (k #2) (2-15) 


6 


are valid. We shall say that a function A(t) can be represented as a linear combination 


of a system of functions 


C(t), C(t), Seo 5 C(t) (2-16) 
if we can write 
n 
A(t) = a a, (t) (2-17) 


where some of the a, may vanish. 
If we assume that the functions (2-16) are orthonormal, then, taking the scalar product 
of both sides of Eq. (2-17) with Cy(t) and expanding, we obtain, with the use of Eqs. (2-12) 


and (2-13) 


Alt) Colt) = ay (2-18). 


We call the coefficients ay the coordinates of the function A(t) in the system (2-16). 
Obviously, the function A(t) is completely characterized by the n coordinates Ayr ere y a, 
if the system (2-16) is specified. In particular, if we take as the system of orthonormal 
functions the system (2-14), we obtain 


oO 


A(t) = i a,I,(t) (2-19) 
0 LL 


where 


a9 = A(t) Tp {t) (2-20) 


The series (2-19) is the familiar expansion of the function A(t) as a Fourier series in the 
interval -T/2, +T/2. According to (2-14), the amplitude of the cosine term of frequency 
m/T is 2 aon? and the amplitude of the corresponding sine term is 2 aom-l? 
If the oscillation A(t) is a signal, then we usually only consider a finite number of 
terms of the sum (2-19), with indices from £ to L,, say, since the components of the 
signal are as a rule so small outside a certain frequency range that they are masked by 


noise or by the components of other signals being transmitted on neighboring frequencies. 


b 


In this case 


2 
A(th= } a, Init) (2-21) 
gin, e+ 
Let ay, » ar be the coordinates of the function A(t) in the system (2-16) and let 
bd, scehs «8 bo be the coordinates of the function B(t) in the same system, Then 


n n n 
Atty) Bt) = | } aC, (t) yb. C,(] = Po apd, (2-22) 
k= k=l k=] 
which follows easily by expanding and using Eqs, (2-12) and (2-13). As a special case, 


7 


we have 


2 a 2 
A(t) = A(t) A(t) = Pay (2-23) 
k=1 
If C(t) is a normalized function with coordinates Cys » c_, then 
n 
yoce 1 (2-24) 
k=1 


Furthermore, if the functions A(t) and B(t) are orthogonal, then according to the formula 


(2-22) and the orthogonality condition (2-10), we have 


n 


iy a,b, = A(t) B(t) =0 (2-25) 
fo) ok k 


The expressions (2-22), (2-23), and (2-25) are the analogs of the corresponding express- 
ions of vector analysis. 


Finally, we show that if two functions 


n 


A(t) = >» a, T(t) 
k=o 


n 
B(t) = b, IL, (t) 


have no components with identical frequencies, i.e. , if for all indices k # 0, either one 


of the a, or one of the by is zero, then 


A(t) B(t) = A(t) Brty (2-26) 


Indeed, under these conditions 


A(t) Bit) = ayb, 


and furthemore 


A(t) =a, B(t) =b 


whence Eq. (2-20) follows at once, 


2-3 Normal fluctuation noise 
SoS eee eres: 


We shall consider noise consisting of a large number of short pulses, randomly distri- 
buted in time, Such noise will be called normal fluctuation noise. This kind of noise in- 
eee ores: 


cludes thermal noise in conductors, shot noise in vacuum tubes, and, in many cases, 


atmospheric and man-made noise as well. Such a noise Process can be represented by the 


expression 


n 
W(t) = > Fy (t-t) (2-27) 
k=l 
where Fy (t-t,) is the k'th pulse occurring in the interval -T/2, +T/2. We assume that 


the pulses are short and begin at the times t Thus 


ke 


Fy (t-t,) = 0 for t< ty and t >t. +o (2-28) 


Here the pulses are to be numbered by the indices k not in the order of their occurrence 
in time, but (say) in order of decreasing amplitude, 

Suppose that the probability of th falling in a subinterval of length dt is dt/T, and 
that it does not depend on the location of the subinterval within the interval -T/2, +T/2 


nor on the other pulses. Moreover suppose that A(t) =0. Then we find that 


Wit) A(t) it FOF (tet) 3 
t) A(t) = = t-t,) A(t) dt = 2-29 
* vr oy err Ons pee 
where 
+T/2 
1 
== F, (t-t,) A(t) dt (2-30) 
e T ade re 


Assuming that dis so small that A(t) changes negligibly in the time J’, we obtain 


At) et Alt,) 
f° ri Beh Ot ae a (2-30) 
k 
where S 
qk =f FY (t) dt (2-32) 
0 


is the area of the k'th pulse. 
The summands Ex are mutually independent random variables. If they are bounded, 
and if the sum of their variances increases without limit as the number of summands is in- 


creased, then, according to probability theory, we obtain in the limit of infinite n 


ee A (2-33) 
VE, Se 


where EE, is the mean value, and DE, = EE ar ae is the variance of the quantity 


KP 9%. is the random variable with distribution law 


2 
1 e* [2 ay 
V2n 


P(x <6, <x + dx) = 


(2-34) 


In what follows, random variables with the distribution law (2-34) will be called normal 


random variables, 


It follews from (2-33) that for sufficiently large n we can write 


n n n 
W(t) A(t) = 2, =@ 5 Dége sy E k 
ee nT a oe k=l § 


Moreover, by (2-31) 


de ot AQ) . 
EE, { —- A(t) Te qk TT 0 


since by assumption A(t)=0. We have also 


2 mu ak Be She aK oo 
ny eg BE “EE, f 7 A°(tL) a Fe ATK) 


a ————— 
Wit) A(t) = Bf AM(t) zr % @ 
k= 


whence 


We shall call the quantity 


the intensity of the process W(t). Thus we have 


\/ A“(t) @, 


Wt) A(t) = 
V2T 
n 


(2-35) 


(2-36) 


(2-37) 


(2-38) 


(2-39) 


(2-40) 


2 
We note that since the sum a” q, is proportional to T, the quantity g*does not depend 


on T, 


We now find W(t) B(t), assuming that 


Blt) =0 and A(t) Blt) =0 


In a way analogous to the above, we obtain 


n 
W(t) B(t) = 
Xk 
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(2-41) 


(2-42) 


where 


Bit) 
Xx = T a, (2-43) 
and 
o 2 
W(t) Bit) = BY“ (t) a, (2-44) 
V2T 
Here 8, 1s a normal random variable which, like 8,» satisfies Eq, (2-34), As shown in 


n n 
probability theory, Z &x and >: xX, are independent random variables in the limit 
k=1 k=l 


n—) oo, provided that 


EEX, =0 (2-45) 
Since we have 
eT{2- 2 2 rf 2 
Ik a a ee 
BE Exe —z Alt.) Blt JF = —s F i A(t) B(t,) dty 
T 7. 
-T/2 -T/2 
a XATOO (2-46) 
= TaD) A t Bit =0 
a 


it follows that the quantities IN and on are independent. We note that this is the case 
and that Eqs. (2-40) and (2-44) are valid even when A(t) #0 and B(t) # 0, if we subtract 
from the process its mean, andif T is sufficiently large. This fact will not be proved 
here, since it is not needed for the subsequent analysis, 

We have called the random variable ® normal if the probability that it lies in the 
interval (x, x +) is given by (2-34). It follows from this definition that the probability 
that 6 > x is given by 


ox 


oo 2 
Ple>x) = —L— | gre te ae u f 
vV20 x V2" Yaw 


2 
ge le ade Ai (2-47) 


The value of this integral can be found in tables. We have introduced the function V(x) 
because it will be very often encountered below. It is shown graphically in Figure 2-1. 


The probability that 6< x is 


P(8 <x) = 1 - V(x) = V(-x) (2-48) 
The mean value of @ is 


Ee =0 (2-49) 


-2 -4 0 4 2 3 4 5 6 7 8 
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The mean value of 9* is 


De = Ee =1 (2-50) 


2-4 Representation of normal fluctuation noise as a Fourier series 
ee ee OR ete f.08. a: S OUTIET Series 


The normal fluctuation noise introduced in Section 2-3 can be represented as the 


Fourier series 


oo 


W(t) = >. Wy I, (t) 2-51 
a ( ) 


(if we neglect the constant component), where 


Wp = Wit) Tp) (2-52) 


If L > 0, but is not so large that the period of the harmonic is of the same order of 


magnitude as the length of the noise pulses, then according to (2-40) 


yo = ey (2-53) 
vet 
Thus, if we pick out from the noise the components with frequencies from fy = f/T to 


fy, = V/T, where M2 1, and Vis not too large, then these components are 


£ L 


2 2 


o 
Ip (t) 89 = I 6 
Vothd Dzh V2T Uy (t L Vit peigs ig () L 
(2-54) 


‘ 2n 21 
(8594 sink T t+ 859 cos L —* t) 


where L, = of) £, =2V. The quantities 89 figuring in this expression are constant 
on the interval -T/2, +T/2, but, as random variables, they can be different for different 
noise samples. It should also be noted that, according to Section 2-3, all the 99 are 
(mutually) independent, since all the Tp (t) are (mutually) orthogonal. 


The process W {t) will be called normal fluctuation noise with constant intensity in 


uv 


? 
the frequency range A/T to V/T. The mean square of this process over time T is, 


according to (2-23) and (2-54) 
Wee (th) =a OD 0, (2-55) 


Averaging with respect to realizations of the process, we obtain for the square of the 


effective value the expression 


2 4, 
2 
Bo Se we y(t) = S LE “9 (26) 
pr? L=eu-) 
and, since Ee = 1, we have 
H2 = & We (t) = o (2y- an) 7 (f - £,) (2-57) 
fev 2T vd 
whence 
2 
EW (i) 
o = — ees eens (2-58) 
yo fy ie 
Thus, O&O is the effective value of the process W y(t), referred to unit bandwidth. 
? 
We now show that if the waveforms A(t) and B(t) satisfy the condition 
A(t). =0, B(t) =0, A(t) Blt) =0 (2-59) 


and can be represented as Fourier series which have no components with frequencies less 


than fy =K/IT and greater than f,,= V/T, then 


- 2 

Wiesy t) A(t = A“ (t) aN (2-60) 
_ ¢ 2 

Wav) B(t) = B’(t) 6B (2-61) 


where aN and 8, are independent normal random variables. Indeed, the process (2-51) 


can be written as 


W(t) = W'(t) + Val) + W'Xt) (2-62) 
where 
2m-2 
W(t) = Wp Ip (t) 2-63 
2, iy) ( ) 
. 
W(t) Tp (t) eae 
2=2v+1 2 


Then, by hypothesis, W'(t) and W"(t) have no components with frequencies coinciding 


with the frequencies of the components of the oscillations A(t) and B(t). Therefore 
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wt) A(t) =0, “Witty A(t) = 0, 
(2-65) 


(t) Bt) = 0, w(t) B(t) = 0 
whence, multiplying both sides of (2-62) by A(t) and B(t) and taking the scalar product, 
we obtain the expressions (2-60) and (2-61), with the help of Eqs. (2-40) and (2-44). 


For simplicity, we shall often consider the random function 


£, : 
@it)= FF Qy Ip (t) = Q' Is (t) (2266) 
gee 24 pe bit 
which differs from the process W y(t) by the constant factor Here we have 
written we vet 


7, = “p+ 2p-2? Iyit) = Brep-2') n = 2u -(2y - 2) (2-67) 


According to (2-60), we have 


@ (t) A(t) x] AM(t) Oy (2-68) 


since Way!) equals @) (t) if ee l. 


V2T 


2-5 Linear functions of independent normal random variables 


We shall now find the linear function 


D 
apy 8 (2-69) 
ye 20 
of the independent normal random variables % ,» where the 29 are arbitrary constants, 
We set 
: 
A(t) = ay I(t) (2-70) 
v= Lh 
n 
® (t) p: Q%p Ip (t) (2-71) 
2 BL 
Then, according to (2-22) 
is 
A(t)@(t)= ¥ ap % (2-72) 
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On the other hand, by (2-68) and (2-23) 


—— | 2 ae 

A(t) @® (t) =7/ A(t) wl) 7 ap 84 (2-73) 
: 7 yA? 2-74 
ee Ro ap aN (2-74) 


whence 


Analogously, we have 


E by % = s bp a5 (2-75) 


Moreover, if 


apbp = 0 (2-76) 
ee 2 2 
then aN and 9, are independent normal random variables; for, writing 
i) 
B(t) = ba a(t (2-77) 
(t) rH oy ( ) 


we find (as above) 


where, according to (2-76) and (2-22), A(t) B(t) = 0, and therefore, by (2-60) and (2-61) 
On and on are independent. 


2-6 The probability that normal fluctuation noise falls in a given region 


We shall say that a function lies in a given region if its coordinates satisfy the 
conditions that define the region. We now find the probability that the function (2-66) lies 


in the elementary region defined by the conditions 
<e< 
My ay Se Ey 
eo er em eo ere ee ee (2-78) 
t 
+ las a6 : dy 


Since the % figuring in these inequalities are independent random variables which satisfy 
the conditions (2-34), the probability that all the inequalities (2-78) are simultaneously 


satisified is 
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dy dy dy 
P(2-78) = ——+ exp(-y*/2) exp(-y5/2) ++» ——2 exp(-y2/2) 
Vex Ven Von : 
dy, dy, «** dy n (2-79) 


n 1 2 
rrr a es 


The probability that the function @(t) will lie in some region, say the region R, which 
can be divided up into elementary regions of the type (2-78), is obviously equal to the sum 
of the probabilities that the function will fall in one of the elementary regions into which 
the region R is subdivided. Since the elementary regions are infinitely small, this sum 


reduces to the integral 


dy, dy . dy, n 
P(@(t) in n= ff : a on exp(- 5 — > ye) (2-80) 
R (2 x) i= 1 


which is to be taken over the values Yprcer a Vy belonging to R. 
n 
In the case where the region R is so small that . yo can be regarded as constant 


in integrating over the region, then the exponential can be taken out from behind the 
integral sign, and we obtain 
-n/2 a 
P(@(t) in R) = (2x) exp(- 5 = zy (2-81) 


i= 


where 


AV =ff--f dy, dy, +++ dy, (2-82) 
R 


Using the terminology of three-dimensional space, we shall call the quantity AV _ the 
volume of the region R. 


If a function 


n 
y(ty= oy, Nt) (2-83) 
i=l 


lies in the region R, then the coordinates of the function can be put into the formula 


(2-81). According to Eq. (2-23), we have 


= n 
r= Eo yy (2-84) 
so that 
Z AV 1 2 
P(@(t) R) = exp(- 5 Y (t)) (2-85) 
@(t) in (ane z 2 


17 


We can draw the following conclusion from this formula. The probability that the random 
function @(t) defined by Eq. (2-66) lies in a small region in which the function Y(t) also 
lies, is proportional to the volume of the region, and depends in addition only on the 
effective value of the function Y(t), decreasing as the effective value increases. Bya 
small region we mean here a region such that the effective value of all functions lying in 


it can be regarded as the same in calculating the integral (2-80). 


2-7 Geometric interpretation of our results 


The results obtained in this chapter, as well as the results which we shall obtain 
below, can be interpreted by using the geometry of n-dimensional space. Although it is not 
very easy to visualize n-dimensional space, still such an interpretation has many advan- 
tages, especially for those who are inclined to think in geometric terms, The point is that 
relations which are valid for any n-dimensional space are valid in particular for the 
special cases of two and three-dimensional space. This allows one to guess and verify 
general properties of spaces with many dimensions with the aid of the descriptive models 
of ordinary geometry. Moreover, the use of terminology and models borrowed from the 
geometry of three-dimensional space allows one to more easily keep in mind the results 
which have been obtained. 

We were agreed to deal with functions in an interval T and with frequencies lying in 


certain bands. In this case, the functions being considered can be represented in the form 


- 
A(t) = I, (t) 
L=2p-1 v L 


where the Tg (t) are specified functions given by Eq. (2-14). Thus, any function under 


consideration is completely determined by n = 2y- 2h quantities a We can represent 


° 
this function conventionally either by a radius vector in n-dimensional space, the terminal 


point of which has coordinates Bone? A2y? » or by the terminal point itself, 


tee ya 
Such a vector will be called the vector i aaeneereyeatr the function A(t) or briefly the 
vector of the function A(t), In the case n= 2, this representation is especially graphic. 

The function Ty (t) has all coordinates equal to zero except the coordinate with index £, 
which equals one, Thus, the radius vector corresponding to Tp (t) lies on the axis indexed 
by L , and has unit length. 

It is not hard to see that the vector of a sum of functions equals the sum of the vectors 
of the individual functions, The vector of the difference of two functions is the difference of 
the vectors of the individual functions. According to the definitions of Section 2-1, the 
scalar product of two functions equals the scalar product of the vectors corresponding to 
them, as follows from Eq. (2-22). Thus, addition, subtraction, and scalar multiplication 


of functions can be replaced by addition, subtraction, and multiplication of their vectors. 
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Furthermore, orthogonal functions correspond to orthogonal vectors, and functions which 
coincide in direction correspond to vectors which coincide in direction. 

The magnitude of the effective value of a function, the square of which is given by the 
expression (2-23), equals the length of the vector corresponding to the function. Accord- 


ingly, the square of the distance between the points which correspond to the functions A(t) 


and B(t) equals (A(t) - B(t))“. A unit vector corresponds to a normalized function, A 
system of orthonormal vectors corresponds to a system of orthonormal functions. The 
notion of the volume of a region, introduced in Eq. (2-82) of Section 2-6, corresponds to 
volume in the space in which we construct the vectors. 

To the random function @(t) defined by Eq. (2-66) there corresponds a random 
radius vector, The probability that the end of this vector falls in some small volume AV 
is given by Eq. (2-85). As is evident from the formula, this probability is proportional to 


the volume AV, and depends also on the distance of the volume from the origin of coordi- 


nates, This distance is equal to the quantity V y(t), It follows from Eq. (2-68) that 
the projection on any direction of the vector corresponding to @(t) is equal to the 
scalar product of the unit vector coinciding with the given direction and the vector corres- 
ponding to @(t), and is always a normal random variable. The projections of the vector 
corresponding to @(t) on orthogonal directions are mutually independent normal random 
variables. Everything which has been said here about the vector corresponding to the 
random waveform @(t) can be carried over to the vector corresponding to the noise 


waveform W ylt), since these waveforms differ only by a constant factor. 


ye 
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PART II 
TRANSMISSION OF DISCRETE MESSAGES 


CHAPTER 3 
THE IDEAL RECEIVER FOR DISCRETE SIGNALS 


3-1 Discrete messages and signals 

In this part we shall consider the transmission of discrete messages, i.e, , of 
messages which can have a finite number of completely specified versions; we shall also 
consider the effect of noise on the transmission of such messages. As already mentioned, 
the category of transmission of discrete messages includes telegraphy, remote-control 
when there is provision for a finite number of distinct commands, various kinds of signal- 
ling, etc. We shall use the example of telegraphy to make more precise what we mean by 
a message. As already noted, in general by a message we mean that which is to be trans- 
mitted. Thus, we shall designate as a message an entire telegram, the separate words of 
which it consists, and the separate symbols of which the words consist. It is also possible 
to call a message the voltage waveform which corresponds to the transmitted word or 
symbol, and which is produced by the telegraph transmitter, to be transmitted to the 
telegraph receiver, This voltage waveform usually consists of individual smaller units, 
which follow one another in sequence. For example, when the five-symbol telegraph code 
is used, the voltage waveform corresponding to one symbol consists of five units. We can 
also regard each of these units as a message. Thus, we can mean by a message both the 
text of the telegram and the elements which go to make it up, as well as the voltage wave- 
forms produced by the telegraph transmitter and their elements, A message can be com- 
plex and can consist of a series of simpler messages which follow one another in sequence, 
For simplicity, we shall assume that in the case where the telegraph transmitter sends 
nothing, a message is sent to the effect that the telegraph receiver should print nothing. 
The receiver has to reproduce the transmitted message, Therefore, in the case where we 
take as the message the separate symbols, words, or telegrams, the telegraph printer 
must be included as a component of the receiver. 

Suppose the system in question provides for the transmission of messages which can 
take on m values. We assume that to each value of the message corresponds a definite 
signal (i.e. , waveform) which acts upon the receiver when the message is transmitted in 


the absence of noise. We designate those m values of the signal by 


A,(t}, Ao(t), «6. , A(t) (3-1) 
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In particular, one of these signals can be zero. Using Eq. (2-21), we can represent these 


£ 


2 
A.(t) = a, Ip(t) (3-2) 
k oe “ke? 
Rhy 


Signals as 


Suppose noise is added to these Signal waveforms, We assume that for all frequencies 
involved in the sum the intensity of the noise is the same and equals 0”. In this case, 


when the signal A(t) is transmitted, the sum voltage acting on the receiver is 


£, 
X(t)= Fo x I(t) =wWw _(t) + A, (t) (3-3) 
= L v k 

Lh, L J» 

where Wa y(t) is the noise waveform defined by Eq. (2-54). In this expression, we sum 
? 

over all frequencies to which the receiver can respond, Using Eqs. (3-2), (3-3) and 
(2-54), we obtain 


Ss 
Q = Vir i) Be FKg (3-4) 


The waveform X(t) acting on the receiver is completely characterized by the coordinates 


Hig gE Shipesie ipo. + 
fy’ 4, +1 b, 


3-2 The ideal receiver 


We shall assume that, depending on the waveform which acts upon it, the receiver 
always reproduces one of the possible messages. It is clear that for every receiver we 
can pick out from all possible values of X(t) (i. e., all possible values of the set 
*g° pi ee *L,) the domain of values for which the receiver will reproduce the 
message corresponding to the signal A) (t). We shall call this domain the domain of the 
signal A, (t). Then, in just the same way, we can pick out the domain of values for which 
the message corresponding to the signal A,(t) will be reproduced, We call this domain 
the domain of the signal Aj (t), and so forth. Itis clear that in this fashion the whole 
domain of possible values of X(t) will be divided into m non-overlapping subdomains. 

Suppose the signal A, (t) was sent. In this case, the waveform X(t) which arrives at 
the receiver in the presence of noise is characterized by the coordinates (3-4), which in 
general can take on arbitrary values, since the 89 are (mutually) independent normal 
random variables. Then there is a finite probability that the waveform X(t) will fall in 
any domain. Assume that it falls in the domain A,(t), where i #k. Then the receiver 
will incorrectly reproduce the message corresponding to the signal A, (t) instead of the 
message corresponding to the signal A, (t). It is clear that the number of correctly repro- 


duced messages depends on the configuration of the domains defined by the receiver. 
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We shall be concerned with the problem of selecting the domains of values of the wave- 
forms X(t), for given signals (3-1), in such a way as to make the number of incorrectly 
reproduced messages as small as possible, or, what amounts to the same thing, in such a 
way as to make the probability of correctly reproduced messages as large as possible, 
The receiver which is characterized by such domains, and which therefore gives the 
minimum number of incorrectly received messages, will be called ideal. To determine 
the configuration of the domains which characterize the ideal receiver, we introduce the 


following notation: 


P(A,) is the a priori probability that the signal A, is sent. 


i) 
Py (X) is the conditional probability that the waveform X(t) with coordinates 
k 


< ae 
¥g, 2 Yg bled ete ¥g, aa nee + ayy (3-5) 


will be received, if it is known that the signal A, (t) was sent. 
PLA) is the conditional probability that the signal A, (t) was sent, if we know that 


the received waveform is X(t), i.e., that it corresponds to the inequalities (3-5). 


n 
P(X)= > P(A.) P, (X) is the probability that the received signal is X(t). 
k=1 k 


With this notation, the joint probability that the signal A, (t) is sent and that the wave- 


form X(t) is received is given by 


P(A,) Pee = P(X) PL(A (3-6) 


x) 


whence 


P(A.) P 4,00 P(A.) P 4 0 
Si ee (3-7) 
P(A,) P, (X) 
na Ag 


If, when the waveform X(t) is received, the receiver reproduces the message corres- 
ponding to the signal A,(t), then the probability of there being correct reproduction when 
X(t) is received is obviously Pi(A,). Similarly, if, when the waveform X(t) is received, 
the receiver reproduces the message corresponding to the waveform Agit), then the prob- 
ability of correct reproduction is P(A). Thus, to obtain the maximum probability of 
correct signal reproduction, when X(t) is received, the receiver should reproduce the 
message which corresponds to the signal for which the quantity PAL) is largest; in 
other words, the receiver should be constructed in such a way as to make X(t) belong to 
the domain of the signal A, (t) for which PY(A,) is the largest. This receiver will 
guarantee the maximum probability of correct message reproduction. No other receiver 


can increase this probability, 
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According to (3-7), when the waveform X(t) is received, the ideal receiver should 
reproduce the message corresponding to the signal which gives the largest value of the 


expression 
P(AL) Py, (X) (3-8) 
k 
The quantity P(A) in this expression has to be furnished; it is determined by the charac- 
ter of the transmitted messages, The quantity Pa (X) is by definition the probability that 
k 


the noise assumes a value which when added to the signal A, (t) gives the waveform X(t) 
satisfying the relations (3-5), It follows from Eq. (3-4) that this probability is the prob- 
ability that the ®% satisfy the inequalities 


V2T V2T 
= (Yp a) wo) <9, < - i) ees Hla 

peed 6 We Bocas & (3-9) 
Y¥2T 


According to Section 2-6 and Eqs. (2-78) and (2-79), the latter probability is 


y dy y 
farrle "hy b, T 2 
Pao (3) a ghle exP “52 =p, ) ~ ag) 


N 


where n = 4, -£, + 1. Moreover, for infinitesimally small sd ioe 5 ede we have 


f, f, 


ch, (yg - als En (xp - ayy)” = (X(t) - ALCO) 


whence it follows that 


dy oo. dy 
P(A,) Py (X) = i a exp - — (X(t) - A(t)? Thi Pay) | (3-10) 
k (n/T)O'S o 


where the larger the value of the exponent, the smaller the value of (3-10). Thus we 


obtain the largest probability of correct message reproduction if we choose the receiver 
in such a way as to make X(t) belong to the domain of the signal for which the quantity 


+T/2 
T (X(t) - A, (ty)? BT nt P(A,) = Je (X(t) - A(t)? dee 6s P(A.) (<1) 


has the smallest value. 
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3-3 Geometric interpretation cf the material of chapter 3 


As we have already remarked, every waveform of finite length and with a finite fre- 
quency spectrum can be represented as a point or radius vector in n-space. Thus, each 
of the m_ signals considered in this chapter can be represented by its own point or radius 
vector, If to the transmitted signal is added a noise waveform with a vector which can 
have an arbitrary direction and arbitrary length, then the resulting received waveform 
X(t) will also be characterized by a point in n-space, which most often will not coincide 
with any of the points corresponding to signals. Depending on the position of this point, 
the receiver will reproduce some message or other. If we combine all the points of our 
space which correspond to received waveforms for which the receiver reproduces the 
message corresponding to the signal A,(t), we obtain the region of space which we called 
the domain of the signal A, {t). Since we assumed that when a waveform is received, one 
of the possible messages has to be reproduced, then every point of space has to fall in 
the domain of some signal, We saw how these domains should be chosen for the ideal 
receiver. In the simplest case, when all the signals are equiprobable (i.e. , when all the 
P(A,) are equal), the domain of the signal A, (t) should consist of points of the space 
which lie closer to the point A, (t) than to any other point representing a signal, i. e., 


points for which 


Sear 
(X(t) = AL ())? < (X(t) = Ag itn)? 


where Ay (t) is any of the possible signals which differs from A, (t). This is natural, 
since the smaller the length of the noise vector, the larger the probability of the noise, 
and therefore it is most likely that the given received waveform was formed by the addition 


of the noise vector to the end of the nearest signal vector. 
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CHAPTER 4 


NOISE IMMUNITY FOR SIGNALS WITH TWO DISCRETE VALUES 


4-1 Probability of error for the ideal receiver 


Even with the ideal receiver there sometimes occurs incorrect message reproduction, 
because of the perturbation of the signal waveform by the added noise. We now find the 
probability of such incorrect reproduction, or, as we say, the probability of error. This 
probability characterizes the noise immunity for reception with the ideal receiver, i. e., 
the optimum noise immunity for the given kind of signals. The probability of error for 
reception with a real receiver can attain this value, but cannot be less than it. 

In this chapter we consider noise immunity for signals which can take on only two 
values A, (t) and A,(t). This case is of great practical interest, since discrete signals 
often consist of sequences of elementary signals, each of which can have only two values. 
According to what was said in Section 3-2, the ideal receiver in this case should reproduce 


the message corresponding to the signal A, (t) if 


ant P(A) <T (X(t) - A,(t))* sof th P(A,) (4-1) 


T (X(t) - Aj)? - 6 
and otherwise the message corresponding to the signal A,(t). 

Suppose the signal A(t) was sent. We now find the probability that the noise (of 
constant intensity in the frequency range AIT to v/T) assumes a value such that the ideal 
receiver reproduces the message corresponding to the signal A,(t). This probability 
equals the probability that the inequality (4-1) is not satisfied when we substitute into it the 
quantity 


X(t) = A, (t) + W oo 


J 


i.e. , the probability that the inequality 


- welt) - 6° in P(A) >T a Aj (t) - A, (t)” - 6 In P(A,) 


is satisfied. Expanding this expression according to the rules introduced in Section 2-1, 


we obtain 


T We (t) - 6% In PlA\) >T Wy vt 427 Wy (A(t) = A, (1) 


pv 


+T (A,(t) - A, (t))* - oO In P(A,) 


whence it follows by Eq. (2-60) that 
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2 
=o? In P(A)) >6 very ae ~A,())* @+T (A,(t) - A, (1) - 0° In P(A) 


P(A,) Vi tiay(o = 4(0)? 


e<sin aay Ba I Oe S = 
y) P(A SSS rz 
1 “/ T (A, (t) - A(t) = 


The probability of this inequality can be determined from Eq. (2-48). Thus, the prob- 


ability that as a result of the addition of fluctuation noise to the signal A,(t), the ideal 


receiver reproduces the incorrect message corresponding to the signal A,(t), equals 
; ~ 2 
P(A, instead of A)) V(a,,) (4-2) 


where we have introduced the notation 


(4-3) 


V rao -ajo? ‘ +T/2 ; LfZ 
| (A, (t) = A,(t))" at (4-4) 


Os sy Ae ee a ee ae 2 


and V is given by Figure 2-1. 
In just the same way, the probability that the receiver will incorrectly interpret the 


transmitted signal A,(t) as A, (t) is found to be 


P(A, instead of A,) = V(a)>) (4-5) 
where 
pb Bg) 
Bio sat ee ln P(A,) (4-6) 


It follows that in the case of two signals the probability of error for the ideal receiver is 


P_= P(A)) V(a,)) + P(A,) V(a 


E (4-7) 


12) 


As is apparent from these formulas, the probability of error, which determines the opti- 


mum noise immunity, depends on two factors--on the ratio P(A,)/P(A,), and on 
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T (Aye AN ae 

a? = ! : 7 Epes ‘h (A,(t) - A (t))* at 
2 

eS 25° 2 


The first factor depends exclusively on the transmitted messages. The second factor 
depends on the ratio of the specific energy of the signal difference to o”, the square of 
the noise intensity, The larger this ratio, the smaller the probability of error, and the 
larger the optimum noise immunity. In this factor, for a given noise intensity, we can 
change only the specific energy of the signal difference. Systems for which this energy is 
the largest afford the best noise immunity, provided that the receivers are sufficiently 
good. 


In geometric terms, both © and the optimum noise immunity are determined by the 


aeanes (A, (t) - A, (t))* between the points representing the signal, and become 


larger when this distance increases. We note also that the probability of error does not 
depend on the expansion interval T, since by hypothesis this expansion interval can be 
taken large enough to have the signal entirely contained within the interval (-T/2, +T/2). 

It also does not depend on the limits of the frequency summation, provided they include all 
frequencies contained in the signals. The probability of error is connected with the mean 
number of incorrectly received messages by the following relation: the number of incorrect- 
ly received messages is on the average equal to NP where N is the number of trans- 


mitted signals. 


4-2 Influence of the ratio P(A,)/P(A,) 
If the probability of transmission is the same for both signals (i.e. , P(A) = P(A) = 


=), Eq. (4-7) simplifies to 


Pe =V(%) (4-8) 


In the case where the noise intensity © is small, so that a@ >> 1, the second term in the 
expressions for a5 and G5 can be neglected, and since P(A)) + P(A) = 1, in this 
case also Eq. (4-7) reduces to Eq. (4-8). In Figure 4-1, curve ! gives the dependence 
of Pr 
the case P(A,)/P(A,) 


respectively. As is evident from these curves, and also from an analysis of the formulas, 


on a for the case P(A,)/P(A,) = 1, and curve 2 gives the same dependence for 


= 10 or 0.1. These curves were obtained from Eqs. (4-8) and (4-7), 
PR gets smaller as P(A,)/P(A,) differs more from unity. In the limit P(A,)/P(A,) = 0 


or 0, we obtain PR = 0, irrespective of a. This result is obvious, since in this case 


the transmitted signal is known in advance, 
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In the case P(A, ) = P(A,), the domains which the ideal receiver assigns to the 
signals A ,{t) and - z(t) do not depend on the noise intensity ©, as follows from Eq. 
(4-1). In i case PA, )# P(A, ), they must depend on o. This means that the regime 
of the ideal receiver mn to ete with o , which in many cases may be inconvenient. 

Let us see how much the probability of incorrect signal reproduction increases if the 
signal domains (i.e., the receiver regime) are taken for the case where P(A)) = P(A,), 
or, what amounts to the same thing, for the case of small o , and are not changed in the 
case where P(A, ) # P(A,) and g is large. For the case where & is small, according 
to (3-11), the receiver domains are chosen so that the received waveform X(t) falls in 


the domain of the signal A, (t) if 


(xy) = A (OF < (xt) - AL)? 


If we repeat the considerations of Section 4-1 for this case, we obtain 


P(A, instead of Ais Via) 
where a is defined by Eq. (4-4). In complete analogy, we have 


P(A, instead of A 


= Via) 


whence the probability of error in this case equals 


Pr 


P(A,) P(A, instead of A)) + P(A,) P(A, instead of A,) 
(4-9) 


V(a) 


since P(A) + P(A,) =1l, Thus, this probability does not depend on the ratio 
P(A,)/P(A,), and equals the probability of error for reception with the ideal receiver in 
the case P(A,)/P(A,) = l, 


Now consider the case P(A,)/P(A = 10 or 0.1. Then, the probability of error for 


2) 
reception with an ideal receiver, specially constructed for this case, is given by curve 2 
of Figure 4-1. On the other hand, the probability of error for reception with the receiver 
just considered is given by curve 1 of the same figure. As is evident from the figure, for 
small a the difference between the two curves can be quite substantial, and in this case it 
is desirable to take into account the unequal probabilities of signal transmission. For 
operation in the region of small oO (i.e. , large a) the difference is small. 

To characterize how much a given receiver approaches the ideal receiver in noise 
immunity, we introduce the concept of the efficiency coefficient of a receiver, which we 
designate by N . By this coefficient we mean the ratio of the signal power for the ideal 
receiver to the signal power for some other receiver under consideration, provided that 


the probability of error and the form of the signals are the same in both cases, Thus, 
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PA) 
P(A) — 
P(Ay) 
PlA2) 
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Fig. 4-1. Probability of error for the ideal 
receiver. Curve 1 is for P(A,)/P(A,) = 1; 
curve 2 is for P(A,)/P(A,) = l0or0.l3 a 
is defined by Eq. (4-4). 
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Fig. 4-2, Dependence of the efficiency coefficient 
on the probability of error for P(A,)/P(A,) = 10 
or 0.1, and for the receiver which is ideal for 


P(A,)/P(A,) = 1. 
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this coefficient shows how much the energy of the signals can be decreased, if we use the 
ideal receiver instead of the given receiver, while keeping fixed the probability of correct 
message reproduction. For the case just considered, this coefficient is equal to the 
square of the ratio of the abcissas of the curves 1 and 2 for the same value of Ph. The 
dependence of on Ph which is obtained in this way : shown in Figure 4-2. As we 

see from this figure, for conditions such that Ph <10°~, and such conditions are common, 


we can take N > 0.9. 


4-3 Optimum noise immunity for transmission with a passive space 
MPH MUM DOTS S LW Oy eee ee ee Pp 


In the case where the signal waveform can have only two values A, (t) and A,(t), and 
one of them, say A,(t), is identically zero, the transmission is called transmission with 
a passive space. If neither of the signals is identically zero, we refer to transmission 
with an active space. For transmission with a passive space, the value of a , which is 


defined by Eq. (4-4) and characterizes the optimum noise immunity, is given by 


(4-10) 
Denoting the specific energy of the signal by 
+T/2 
Qo =T A(t = J a“ (t) dt (4-11) 
-T/2 
we have 
Q 
Se (4-12) 


v2 
Thus, in this case the optimum noise immunity depends only on the signal energy and is 
completely independent of its shape. The larger the signal energy, the larger the optimum 
noise immunity. However, it should not be inferred from this result alone that the use of 
new signal forms and the improvement of receivers cannot raise the noise immunity of 
systems with a passive space. In fact, the noise immunity of systems now in use may be 
much less than the optimum noise immunity obtained above. When this is the case, itis 
clear that both the improvement of receivers and the use of new signal forms, which 
facilitate this improvement, can increase the noise immunity, and in the most favorable 
cases, make it approach the optimum noise immunity. To clarify this matter, in the next 
two sections, we consider an example of a real system with a passive space, and we find 


out how close its noise immunity is to the optimum. 
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4-4 Optimum noise immunity for the classical telegraph signal 


As an example of transmission with a passive space, we consider the case of the 


classical elementary telegraph signal, which we shall take to be 


Ay (t) aU, cos Wot, for O0<t<@% 


(4-13) 


v 
ro) 
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AL(t) 


and 
A,(t) = 0 (4-14) 


To determine the optimum noise immunity in this case, we can take a from Eq. (4-14). 
According to Appendix A and Eqs, (4-11) and (4-12), the quantity Q, which appears in 
this expression is given by 
v, 
1 2 1 ,,2 
C= 5 li Uv dt = 5 Ul Y% (4-15) 


fe) 


Therefore, for the kind of transmission under consideration, we have by (4-12) 


a %% 


a= ay. (4-16) 


According to Section 4-1, we can use the quantity a to determine the probability of error 


which characterizes the optimum noise immunity for this case. 


4-5 Noise immunity for the classical telegraph signal and reception with a synchronous 


detector 


Suppose to receive the signals considered in the previous section, we use a real 
receiver, such that the signals first go through a filter with a pass band from 
w,-2 w, + 
Sr. oe eae 


detector output then enters a device which reproduces the message corresponding to the 


» and then enter a synchronous detector. The waveform at the 


first signal if the voltage on its terminals at the time %,/2 exceeds a certain value, 

and otherwise reproduces the message corresponding to the second signal. Such a process 
occurs, for example, when the voltage is rectified and used to activate a telegraph 
apparatus which operates on time division, 


Regarding the filter as ideal, we have 


U 
us = — {sitlt - SifL(t- %,)] cos wt (4-17) 
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for the signal at the output, as can be obtained with the use of the Fourier integral, if 


retardation in the filter is neglected. In this formula, Si denotes the sine integral, given 


by 


ee -[ S02 ae (4-18) 


Clearly, the noise voltage after the filter consists of components with frequencies from 
ae Nc aaa ae a 


(B-6) of Appendix B, this process can be written as 


, and has conStant intensity @ in this band. According to 


Vag ae nt) cos wit V2 Wi allt) sin wt 


where Wi nit) and ee (t) are independent normal fluctuation noises, the components of 
which ae pene ar from 0 ton/T =f U/2x , and have constant intensity @ in this band. 


Thus, if the signal A(t) is sent, the sum voltage after the filter is 


U 
uy ie Wa vt) = ore (Sift - SiML(t- % )) +~v2 a nt) cos wt 


2 Lee n't) sin ww t (4-19) 


The synchronous detector, as is well-known, gives at its output a voltage proportional 
to the amplitude of the component which coincides in phase with that of the received signal, 
and does not respond to the component which is 90° out of phase with the received signal. 
As stipulatec, the output device reproduces the message corresponding to the signal A, (t) 
or A,(t), depending on the value of the voltage at the detector output at the time t = %,/2. 
Designating this value by Ua we obtain 


uy A 


Ul = = 2Si + 2 we ale, /2) (4-20) 


If A, (t) is transmitted, and 


Uy = V2 Wate?) (4-21) 


if A, (t) is transmitted, i.e. , if no signal at all is transmitted. The value of aie nl® /2) 


is a random variable; according to Eq. (C-2) of Appendix C, it can be expressed as 


ai n(%,/2) = OND/2x 8 (4-22) 
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where ® is anormal random variable. We assume that the output device reproduces the 


message corresponding to the first signal if 


(4-23) 


i, e., if Uy is less than half the rectified signal voltage at that moment, and otherwise 
reproduces the message corresponding to the second signal. 
We now find the probability that the second message is reproduced instead of the 


first, i.e. , the probability that U, does not satisfy the inequality (4-23). This probability 


d 
is 


P(A, instead of A))= P(US<U,) = P(6< -B) (4-24) 
where we have introduced the symbol 
s fi 1/2 o. 
B= (Ul) A/D sie, /2) (4-25) 
according to Eq, (2-48), this means that 


P(A, instead of A) = v(B) (4-26) 
Similarly, we have 
P(A, instead of A,) = P(U') >U,) = P(é > B) = v(B) (4-27) 


It follows from Eqs. (4-26) and (4-27) that 


PE = V(B) (4-28) 


for the given means of reception. 

To obtain the minimum probability of error, we must try to make B as large as 
possible, The dependence of B on the filter bandwidth {b/s can be found by first 
rewriting Eq. (4-25) as 


1/2 Six 


x 


B= (U/0) (%,/2") (4-29) 


where 


x =, /2 


33 


The dependence of Si x/~~x on x is shown in Figure 4-3. As can be seen from the figure, 
this quantity has its maximum value of 1.14 for x = 2. 1. Hence, for the given means of 


reception, the optimum filter bandwidth is 

MLin = 4.2/n%, = 1.34/¢, (4-30) 
and the maximum value of B for this bandwidth is 

> ae = 0.455 U,VG/6 = 0.91 Q/V2 & (4-31) 


Thus, for the case considered in this section, the probability of error is determined by BR; 
just as in the case of ideal reception it is determined by a, in accordance with (4-8). 
Comparing (4-31) and (4-16), we see that Pixs is somewhat less than a@ , which means 
that even when the bandwidth is optimun, the means of reception we are considering gives 
somewhat larger error probabilities than would be obtained with the ideal receiver. 

We now find the value of the efficiency coefficient (introduced in Section 4-2) for the 
means of reception under consideration. Clearly, in the present case, when the bandwidth 


is optimum, this coefficient equals 
(6. ./% > = 0.83 (4-32) 
max ° 


Thus, by using the ideal receiver, the signal energy can be lowered by a factor of 0.83 
while keeping the same probability of error. It follows from this that the means of 


reception in question is very close to being ideal in its noise immunity. 


4-6 Noise immunity for the classical telegraph signal and reception with an ordinary 


detector 


We now consider the probability of error in the case where an ordinary detector is 
used instead of a synchronous detector in the receiver analyzed in the preceding section. 
In this case, the rectified voltage depends on the amplitude of the waveform which is the 
sum of the signal and noise at the filter output. Suppose that the receiver reproduces the 
first message if at the time %,(2 this amplitude U; at the filter output exceeds half 


the signal amplitude, i.e. , if 
U, >(U,/ 2) SiLLG,/2) = UL (4-33) 
and reproduces the second message if this inequality is not satisfied. Rice has calculated 


the probability that the amplitude of the sum of a sine wave and random noise is less than 
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Fig. 4-3. 


a given valde, Using his results, which he presented in the form of curves, we can 
calculate the value of the probability that at the time G,/2 the amplitude of the sum of 

the signal and the noise is less than UL) i.e., that an error occurs. We designate this 
probability by P(A, instead of A,). Then, we find the probability that the noise exceeds 
the value UL in the absence of any signal disturbance, i.e., that the signal A,(t) is 
taterprered as the signal A(t). This probability has been given by many authors including 
Rice , and is the integral from 0 to U. of the Rayleigh distribution 


P(A, instead of A,) = exp(-U“/2H*) (4-34) 


2) 
where H is the effective value of the noise, which, according to (2-57), is ovViL/x in 


our case. Substituting this value and the value of UL into (4-34), we obtain 
; 2 
P(A, instead of A,) = exp(-8 /2) (4-35) 


where @ is defined by Eq. (4-25). 
Assuming that the signals A, (t) and A,(t) are equally likely to be transmitted, we 


have for the probability of error 


Pe = 0.5 P(A, instead of A,) + 0.5 P(A, instead of A,) 
This probability is plotted as curve 3 in Figure 4-4 for the case of optimum bandwidth 
(given here also by Eq. (4-30)), Moreover, for comparison, we have indicated the prob- 
ability of error for the case of reception with a synchronous detector (curve 2) and for the 
case of the ideal receiver (curve 1). In all cases we take as the abcissa the quantity 
Q,/% Oo, where of is the specific energy given by Eq. (4-15). In Figure 4-5, curve 1 


shows the dependence of the efficiency coefficient on P for the kind of reception analyzed 


EF?’ 
in this section. In this case, the efficiency coefficient is the square of the ratio of the 

abcissas of the curves 1 and 2 in Figure 4-4, taken at a given valve of Po The straight 
line labelled 2 in this figure shows for comparison the value 0. 83 of the efficiency coeff- 


icient for reception with a synchronous detector. 


4-7 Results on the noise immunity of systems with a passive space 
As just shown, the optimum noise immunity for constant noise intensity with this 
means of communication is completely determined by the quantity 


ss + co 
of a7 Aj (t) ‘i Att) dt 


1. = oO. Ree "Mathematical analysis of random noise", Bell Syst. Tech. J., 1944-5, 
ect, 3-10. 
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Fig. 4-4. Probability of error for the 

signal with rectangular envelope. Curve 

1 -- ideal receiver; curve 2 -- synchronous 
receiver; curve 3 -- ordinary receiver; 

Q, is defined by Eq. (4-15). The two signals 


1 
are equiprobable, 


Fig. 4-5. Efficiency coefficient for the signal 
with rectangular envelope. Curve 1 -- ordinary 
receiver; curve 2 -- synchronous receiver. 


The two signals are equiprobable. 
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i.e. , by the signal energy. The shape of the signal does not affect the optimum noise 
immunity. Using the classical telegraph signal and receivers with optimum bandwith 
(which were considered in Sections 4-5 and 4-6), we obtain a noise immunity which is 
quite close to optimum, and which is somewhat larger for the synchronous detector than 
for the ordinary detector. Thus, it follows that the application of methods of reception 
which differ from those considered in Sections 4-5 and 4-6 cannot substantially increase 
the noise immunity, when the signal energy is kept constant. This is the state of affairs 
provided that the shape, magnitude, and time of arrival of the signal are known, and the 
noise is of the normal fluctuation type. However, it cannot be inferred from this that one 
should always use the means of transmission and reception discussed here when dealing 
with telegraphy with a passive space. In many cases it can happen that other means of 
transmission and reception are more suitable, for example, because of the influence of 


fading, impulse noise, etc. 


4-8 The optimum communication system with an active space 


In this and the following sections, we shall consider the optimum noise immunity of 
systems in which the signals can take on two values A, (t) and A,(t); neither of which is 
identically zero. First of all, we try to find the optimum system, i.e. , the one which 
furnishes the largest possible noise immunity for a given signal energy or To do this, 


it is clear that we should select signals for which the quantity 


3 +T/2 
1 j 2 1 Was Gaga eee 
a =s-s (A, (t) - A5(t))" dt = ——— T (A, (t) - A,j(t)) (4-36) 
20° Sis 1 2 26° 1 2 


which determines the optimum noise immunity is a maximum under the constraints 


me At(t) at = T A(t) <Q’, i asi) at = T Az) < Q? (4-37) 
-T/2 -T/2 
Since we have 
+T/2 +T/2 +T/2 +T/2 
(A(t) - A, ())% at = 2 J ACtt) dt +2 { adit) des J (A(t) + Aj(t))* at 
-T/2 -T/2 -T/2 =T /2 


to obtain the maximum of this expression we must make the first two integrals as large as 
possible and the last integral as small as possible. The maximum value of the first two 


integrals which is consistent with the conditions (4-37) is obtained by taking 


ne 5 +T/2 , 
Av(t) dt = A5(t) dt = of 4- 
fe. 3 a Z ee) 
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The third integral cannot take on negative values. Therefore, when 


A, (t) = - A, (t) (4-39) 


it assumes its minimum value of zero. There is no contradiction between (4-38) and (4-39} 

Thus a, and therefore the optimum noise immunity, is a maximum if A, (t) and 
A,(t) are equal in absolute value, opposite in sign, and have the maximum permissible 
signal energy. The shape of the signals has no influence on the optimum noise immunity, 
and can be arbitrary. For this optimum state of affairs, it is clear the quantity a , 


which determines the optimum noise immunity is equal to 


+T/2 1/2 
eee J Ar(t) at = JZ Ole (4-40) 
C .T/2 


This value determines the optimum noise immunity which can be obtained for operation 
with an active space and an arbitrary system with two discrete signals, provided that the 
maximum signal energy is specified. Comparing the value of a just obtained with the value 
of a for operation with a passive space (as given by Eq. (4-12)), we see that in the optimum 
case we can decrease the specific signal energy Q? by a factor of 4, while keeping the 
same value of a, and consequently the same probability of error. 

Suppose that for the signal A, (t) in the optimum system we take the signal given by 
Eq. (4-13), and use for the receiver an optimum bandwidth filter (given by Eq. (4-30)), 
a synchronous detectcr, and an output device which reproduces the first signal if the vol - 
tage on its terminals at time %,/2 is positive, ard the second signal if the voltage is 
negative. Then, the efficiency coefficient for reception is N = 0,83, as in the case con- 
sidered in Section 4-5, This is easily seen by repeating in the present context the consid- 
erations of Section 4-5, We note that for this method of reception the noise immunity is 


close to the optimum. 


4-9 Noise immunity for frequency shift keying 


By frequency shift keying we mean transmission which uses the signals 


= < < 
A, (t) = US cos(@,t +9), for O<¢t oe. 
A, (t) = 0, for t<0O or t > %, 
(4-41) 
= <t< 
A, (t) = U, cos(@,t + PY) for O<tS %, 
A, (t) = 0, for t<Qor t> %, 
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For this communication system 


+T/2 
a = =e i (A(t) A, (t))? dt 
2a" Mio 


1 l 
1 +5 cos(2w)t + 20,) i cos(2W5t + 20.) - cos Cw, + W,)t + 0, + 0, j 


" 
Nn 
i ° 
O22 


- cos ((@, - u),)t + 0, - 5] dt 


In doing the integral, the second, third, and fourth terms in the curly brackets give 
quantities which go to zero as Ww) and w, increase; we assume that a) and w, are 
large so that these terms can be neglected, Then after integration and some manipulation, 


we obtain 


2 : a i sinl(W, -W,)%, + a, - 0, = sin(®) - 0.) hess 
o- (w, - WE, 
where 
2 
U- % 
2 oO 
Q° = a ae (4-43) 


is the specific energy of the signals. The value of a so obtained depends on 8, - 9, ; the 
difference of the initial phases. If the frequency shift keying is produced by changing the 
circuit parameters of an oscillator, then 0, = 85; and the expression simplified. In this 


case we have 


Byes Pe eee (4-44) 
o (a, =) % 


The dependence of the expression in curly brackets on (w, - )%,, is shown in Figure 4-6. 
We can draw the following conclusions from an examination of this figure. 
1, For the kind of operation in question, the largest optimum noise immunity is 


obtained for the frequency difference 


(W, -a) 0.7 
ey aaa 3 


(4-45) 


For smaller differences, the optimum noise immunity becomes smaller. This circum- 
stance allows one to determine the minimum frequency bandwidth below which one should 


not go if one wishes to avoid loss of noise immunity. 
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Fig. 4-6, The term in curly brackets in Eq. (4-44). 


2. For the kind of operation in question, and for the optimum frequency difference, 
2 


the value of a is 1. 207 /6 , i.e., 2.4 times larger than the value obtained for trans- 
mission with a passive space, if in both cases the specific signal energy Q is identical, 
Thus, the optimum noise immunity for frequency shift keying is not much larger than the 
optimum noise immunity obtained for the operation with a passive space analyzed in 
Section 4-4, Moreover, if we bear in mind that in the latter case, according to Sections 
4-5 and 4-6, we can come very close to the optimum noise immunity, then we are led to 
the conclusion that we cannot get appreciably more noise immunity with frequency shift 
keying (in the case of undistorted signals and normal fluctuation noise) than with classical 
amplitude modulation, The gain in noise immunity which is observed when changing 
from amplitude to frequency mcdulation (for short wave operation) must evidently be 


ascribed to signal distortion produced by fading. 


4-10 Optimum noise immunity for normal fluctuation noise with frequency-dependent 


intensity 
Until this section, we have considered normal fluctuation noise consisting of a large 
number of very short pulses which have a constant intensity. In Appendix Dit is shown 
that noise consisting of pulses of arbitrary shape can be written as 


2n : * 
T it + 953 


cos ox. 9] (4-46) 


yang» 
if we take into account components with frequencies from felt to WV/T; here the 0* are 
(mutually) independent normal random variables. This expression differs from (2-54) in 
that here the amplitude of a noise component depends on its frequency. We now explain 
how the case of the noise (4-46) can be reduced to the case considered previously. 

Suppose that the received signal can again take on two values A 1 {t) and A z(t), and 
suppose that to the signal is added the noise W* y(t) with the pee o*(f), es varies 
with the frequency. We use the receiver R ees according to the scheme shown in 
Figure 4-7a, In this scheme B designates an equalizer, i.e., a linear device which has 

k 


k(f) = 2 (4-47) 
o*(f) 


for the amplitude of its transfer function, where ko is a constant. The phase of the 

equalizer transfer function can be arbitrary. In going thyedeh the equalizer, the noise 
will be altered; instead of the noise W* 2) with intensity os (f) acting at the point 1, 
we obtain at the point 2 the noise W 


Av 
fluctuation noise, but which has the intensity 


iy which (according to Appendix D) is also normal 


5 o*(f) k(f) 
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Fig. 4-7. Ro is the ideal receiver for the signals 
Ayo (t) and Aj 9 (t) and noise with constant intensity 
o.3 R is the ideal receiver for the signals A, (t) 
and A,(t) and noise with intensity o*(f); B is 
the four-pole with transfer coefficient K, where 


|K| = oi /o* (8); Bo} is the four-pole with 


transfer coefficient K™ } : 
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i.e. , constant intensity. The signals also change their form in going through the equalizer; 
let them have the forms Aj oft) and Ajo(t) at the point 2, Clearly, the receiver R pro- 
duces an error if and only if the receiver Ro produces an error. Thus, the probability 
of error of the receiver R for the signals A, (t) and A,(t) and for noise with intensity 
o*(f) equals the probability of error of the receiver Ro for the signals Aj oft) and 
Ajo (t) and for noise with intensity o.- Thus, in order for the receiver R_ to give the 
smallest probability of error, it is clear that we should choose the receiver Ro to be 
ideal in the sense of section 3-2, 

The receiver R just obtained is ideal for reception of the signals A, (t) and A,(t) 
in the presence of noise with intensity o-*(£). Indeed, it gives the least possible probability 
of error for receivers constructed according to the scheme of Figure 4-7a, and moreover, 
this scheme can be used to construct a receiver with the same probability of error as any 
other receiver. In fact, an arbitrary receiver R' (see Figure 4-7b) is equivalent to the 
receiver shown in Figure 4-7c, where Be: designates the linear four-pole inverse to the 
four-pole B, and the receiver with the diagram shown in Figure 4-7c reduces to a receiver 
with the diagram shown in Figure 4-7a. The optimum noise immunity is characterized by 
the probability of error of the ideal receiver just obtained, Obviously, this probability of 
error is that of an ideal receiver for the signals Ajo (t) and A(t) and noise with the 
intensity ome which is independent of frequency. The latter probability can be determined 
by the formulas of Section 4-1, if in these formulas we replace A, (t), A, (t) , and go- 
by Ajolt), Ayzo(t), and o> respectively. 

In this section we have examined a method which takes into account variable noise 
intensity for the case of two discrete signals, This method also works for all the other 
cases considered below. Therefore, we shall hereafter not be concerned any more with 


this matter. 


4-11 Geometric interpretation of the material of chapter 4 


In the case of two discrete signals, the domains of the ideal receiver which correspond 
to these signals are determined by the inequality (4-1). If instead of the inequality sign, 
we write an equality, then the points corresponding to the waveforms X(t) defined by this 
equality will form a plane. This plane, which is perpendicular to the line joining the signal 
points A, (t) and A,(t) is a boundary plane, dividing the domains of the signals A, (t) 
and A,(t). In the case where the signals are equiprobable, the plane passes through the 
midpoint of the line joining A, (t) and A,(t). An error occurs when the noise vector 
adds to the radius vector of the transmitted signal and gives a resultant vector with a 
terminus lying on the other side of the boundary plane, Since all directions of the noise 
vector are equiprobable, it is natural that the probability of error depends only on the 
distance of the boundary plane from the signal points, or equivalently, on the distance 


between the signals, i.e. , on the quantity (A, (t) - A,(t)) » as already shown, 
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In the case of transmission with a passive space, the radius vector of one of the 


signals is-zero. In this case, the optimum noise immunity depends only on the quantity 


At (t), which determines the distance between the end of the radius vector of the signal 


A, (t) and the origin of coordinates. To find the optimum communication system (Section 
4-8), we posed oursleves the problem of finding the system which uses two signals with 
radius vectors not exceeding a certain quantity in length, and which has the maximum 
distance between the ends of these vectors, Naturally, such a system is obtained by 
taking radius vectors of the maximum possible length, and then orienting them in opposite 


directions, i.e. , by setting one of the vectors equal to the negative of the other. 
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CHAPTER 5 


NOISE IMMUNITY FOR SIGNALS WITH MANY DISCRETE VALUES 


5-1 General statement of the problem 


In the preceding chapter we considered the noise immunity for the case where the 
signal can take on only two values. In this chapter we are concerned with a similar 
question, but for the more general case where the messages and therefore the signals can 


have m_ discrete values, Let these signal values be 
Aj (t), A(t), --. 5 A (t) (5-1) 


and let us find the probability of error for the reception of such signals with the ideal re- 


ceiver considered in Chapter 3. This probability will obviously characterize the optimum 


noise immunity. 
Suppose the signal A, (t) was sent. Then the waveform acting upon the receiver is 


X(t) = Mav + A, (t) (5-2) 


Then clearly the receiver will reproduce the message corresponding to the transmitted 


signal A, (t) if (according to Section 3-2) we have 


T (X(t) - A,(t)* - G7 In P(AY) < T (X(t) - A,(M) - o In P(A,) (5-3) 
for all j = 2,... .,m. An error occurs if even one of the inequalities (5-3) is not 


satisfied. Substituting the value of X(t) from (5-2) into Eq. (5-3), we obtain after some 


manipulation 
es ee 
2T W Pricer = A, (t)) T (A. (4) = A, (t)) +o In PY (5-4) 


The probability that this system of inequalities is fulfilled is the probability of correct 
reception of the signal A, (t) with an ideal receiver. Similar relations obtain for the 
other signals. In the general case, the size of this probability is given by integrals which 
are not evaluated. Therefore, in what follows we shall examine only the most interesting 


special cases, 


5-2 Optimum noise immunity for orthogonal equiprobable signals with the same energy 


We now consider the case where 


T A(t) = Q%, A(t) A,(t) = 0, P(A,) = = (5-5) 
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for i,j} = 1,2,... ,m, but i # j. In this case, using (2-60) and (2-61), we easily 
reduce the inequality (5-4) to 


o V2T Art 0, - o WIT At(t)@, <T (AF (t + At(t)) 


9; 420. 2S V2 Q/o (5-6) 


for j = 2, 3,... ,m, where the or are (mutually) independent normal random variables. 


Suppose aT satisfies the condition 
y Se x y + dy - 
1 (5-7) 


According to (2-34), the probability of this is 


a 2 
2 exp(- 5) a 


Ven 
In this case, for the j'th inequality (5-6) to be satisfied, we must have 
2 Q 
< 2 2 
a, r ammeiae (5-9) 


According to (2-48), the probability of this is 


1 meas) (5-10) 


Since all the Be are independent, the probability that all the m-1 inequalities (5-6) and the 


inequality (5-7) are simultaneously fulfilled is 
2 m-1 
—SY— exp (- 5) [1 -v (2 2..5)| (5-11) 
Y2n o 


From this it follows that the probability that all the inequalities (5-6) are satisfied for 


arbitrary 8, is 


+ m-1 2 
P(A, correct) = : 7 1-V (a2 + ») exp(- x) dy (5-12) 
fin ie o 


This probability is the probability that the transmitted signal A, (t) is correctly inter- 
preted by the ideal receiver. In the case under consideration, this probability is the 


same for all the symbols and characterizes the optimum noise immunity. 
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5-3 Example of telegraphy using 32 orthogonal signals 


On the basis of the theory presented in the preceding section, we now calculate the 
optimum noise immunity for the case of telegraphic sar ad where the signals 
characterizing the separate letters all have the same energy Q and are orthogonal to 
one another. This will be the case if the letters are transmitted as sine waves which 
have the same amplitude U. and duration T. for all the letters, but a different 
frequency for each letter. Under these conditions, the waveform representing the k'th 


letter is 


" 


A, (t) US cos(@, t + 0) for O<t< =. 


(5-13) 


U 
oO 


A, (t) = for t<0O or t>%, 


The scalar product of the waveform A,(t) and the waveform A, (t) corresponding to the 


i'th letter is equal to 


+T/2 
_ il 
Atty Aj) = 5 ) A(t) A, (t) dt 
-T/2 
ue sin(@)- w)%, sin ((w, +9.) %, + 20] - sin 29 
So ee a 5-14 


If we assume that (W,. - Wi)G/2n and (a. +W.)U [2x are integers, then the waveforms 
A, (t) and A, (t) are orthogonal, since the expression (5-14) will then vanish. Obviously, 
these waveforms can also be regarded as orthogonal if (wd, - A es is an integer and 
ww, >>| w,. -w; | . Thus, the signals are orthogonal if their frequencies are separated 
from one another by multiplies of 1/%; and if their sums are much greater than their 
differences. The signals are also orthogonal if the separate letters are transmitted as 
arbitrary waveforms which do not overlap, for in this case obviously 

+T/2 

aac 4 
-T/2 


A, (t) A(t) dt = 0 (5-15) 


since for any t at least one of the factors inside the integral vanishes. Furthermore, if 
we assume that the probability of transmission of each of the signals is the same and that 
m = 32, then by numerical integration of Eq. (5-12), we obtain the result represented by 
curve 1 in Figure 5-1, where the quantity Q/O is plotted as the abcissa, and the prob- 
ability Pr =l- P(A, correct) is plotted as the ordinate. Later we shall compare the 
probability of error obtained in this way with the probability of error for other means of 


communication, 
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Fig. 5-1. Probability of error with 


the ideal receiver for 32 signals 
with the same a priori probability. 
Curve 1 -- transmission of ortho- 
gonal signals; curve 2 -- trans- 


mission of five two-valued pulses. 


49 


5-4 Optimum noise immunity for compound signals 
¥ptimum pore ney Se 


Very often complicated signals consist of a sequence of simpler signals. Thus, for 
example, in telegraphy the signals corresponding to the letters and characters almost 
always consist of separate two-valued elementary signals which follow each other in 
sequence and have the same length. We now find the optimum noise immunity for such 
signals. We begin with the general case. 

Suppose that the first elementary signal which makes up the compound signal can have 


one of the following values: 


Bis Botan Bt) (5-16) 


Suppose that the second elementary signal begins at a time Y after the beginning of the 


first signal, Then it will obviously have one of the following values: 


B, (t- 7), Ba(t- %), deb AG Ba(t- ©) (5-17) 


Finally, the £'th elementary signal will have one of the following values: 
By (t-£T+%), Bo(t-LT+%), ... » Bii(t-LE+ ©) (5-18) 


If the compound signal consists of n elementary signals, then clearly it has the following 


form: 


B.., (t) + By a(t- &) tisee + Belt -n@%+@) (5-19) 


where ky, Boe KO are certain integers which can take on values from 1 tom, depending 
on which compound signal is sent. In the case under consideration, the compound signal 
can have m” values. We assume that the separate elementary signals which follow each 
other in sequence do not overlap. Under these conditions B. (t-k@) and B i(t- £2) will be 
orthogonal for arbitrary k # L » as was shown in the preceding section. 

We now find the probability of error for the compound signal considered here, when it 
is received on an ideal receiver. Obviously, for the compound signal in question to be 
received without error by the ideal receiver, it is necessary and sufficient that all the 
elementary signals of which it consists be received without error by the receiver. We 
now show that errors in the separate elementary signals are independent under the con- 
ditions being considered and for reception with the ideal receiver. In fact, according to 


Section 5-1, if the 'th elementary signal has the form 


B. (t- LE4 %), 


then it will be received without error on the ideal receiver provided that the noise has 


values such that the random variables 
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Wt) (B.t- 24+) - Bt- Lee) j= - 
yay F U+E jMt- LE+% j 1, 2,...,m _ (5-20) 


have values satisfying the inequalities 


vA 
2T Wav”) [Bi(t-£0+%) - Bi(t-£U+%)) < T [Bi(t-£E + %) - B.(t-£E+%)] 


P(B.) 


2 
+o l1n (5-21) 
P(B. 
PIB 


Moreover, the k'th elementary signal, which we assume has the value 
B,,(t- kt +%) 


will be received without error if the random variables 


: = = : a ™. 
VEN [ Bt kT+T) B. {t kt+%)] (5-22) 


have values satisfying the inequalities 


Se ee ee i ae ee ee eee ere 
ray Wy") [Bj(t-kU + %) - B, (t-k%+%)] < T Beck +) - B.(t-k&+B)] 


Ea: ani, eee eee (5-23) 


Since the functions in the square brackets in the expressions (5-20) and (5-22) are ortho- 
gonal, then, according to Section 2-4, these expressions are mutually independent random 
variables, which means also that the inequalities (5-21) and (5-23) are satisfied indepen- 
dently of each other. This proves the statement made about the independence of the error 
probabilities of the separate elementary signals. 

The probability of correct reception of each elementary signal can be determined by the 
methods presented earlier. Obviously, in the case being considered, these probabilities 
are the same for all elementary signals (we assume that their a priori probabilities are the 
same) and are denoted by P (corr. elem.). Since as remarked, errors in the separate 
elementary signals are independent of each other, then obviously, the probability that all 
n elementary signals which form one compound signal are correctly received, i.e., that 
the compound signal is correctly received, has the form 


1 - Ph = {P(corr,. elem.) ]” (5-24) 
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5-5 Example of a five-valued code 


We now apply the theory of the preceding section to f.m,. telegraphy using a five- 
valued code. In this communication system, the signal corresponding to one character 
consists of five elementary signals, which follow one another in sequence, and each of 
which has the form discussed in Section 4-9. We shall assume that the probabilities of 


both values of the elementary signal are the same. In this case 


P(error elem.) = V(@) (5-25) 
where a is defined by Eq. (4-44), and 


Bes a2/e* (5-26) 


where a is the energy of the elementary signal, provided that the frequency difference 
is such that the term in curly brackets in Eq. (4-44) equals unity. For this means of 
communication, the signal can have m” = a = 32 different versions. 

We now compare the optimum noise immunity for the signals in question with the noise 
immunity for the orthogonal signals studied in Section 5-3, which can also have 32 differ- 
ent versions, To do this, we express org in terms of the specific energy of the whole 


signal by writing Q* = 5Q°. We obtain 


a” = 0.207 /6% (5-27) 
whence, according to Eqs, (5-24) and (5-27), the probability of error of the compound 


signal is 


P= 1- (P(corr. elem.)]" = 1- [1 - V(V0-2 Q/o))° (5-28) 


The value of this quantity is given by curve 2 in Figure 5-1, where Q/o@ is plotted as the 
abcissa and Ph as the ordinate. Comparing this curve with curve 1 of the same figure, 
which gives the probability of error for a similar system with orthogonal signals, we see 
that the orthogonal system is more advantageous. To obtain the same probability of error 
with the orthogonal system, we need a signal energy as much as 3.5 times less than with 
the compound signal. On the other hand, the bandwidth occupied by the eoripeaude: pial 


is approximately 3 times less, since in this case instead of 32 frequencies we need 
transmit only two frequencies, and moreover, the signals on these two frequencies have 


to be only 5 times shorter than in the case of the orthogonal signals. 
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5-6 The optimum system for signals with many discrete values 


We now find the optimum system containing m signals, just as in Section 4-8 we 
found the optimum system containing two signals. Suppose we have a system of equi- 


probable signals 


A,(t), Ag(t), -.+ 5 AL (t) (5-29) 


We shall show how to decrease the average energy of these signals without changing the 
optimum noise immunity. The optimum noise immunity is defined by the probability that 
the inequalities (5-4), which involve the signal differences, are satisfied. Thus, if we 


introduce the new signals 
t = - 
Ay (t) A, (t) + B(t) (5-30) 


then the inequalities (5-4) are not changed, which means that the optimum noise immunity 
for the signals A,.(t) and AL (t) is the same. Geometrically, this means that if all the 
points corresponding to the signals undergo parallel translations by the same amount, 
then the distances between them and the optimum noise immunity determined by these 
distances, do not change. 


We now find what the waveform B(t) should be to make the average signal energy 


2 > fee Te 
ae = YE a(t) (5-31) 
k=l 
a minimum, Using (5-30), we obtain 
Th m 
2 L 2 2 
Qy -= fe Ajit) + 2 Be a,(o| B(t) +m B (t) (5-32) 


— 


If we change B(t} while keeping B“(t) constant, then the first and last terms in the curly 
brackets do not change, and a minimum is obtained when B(t) has the opposite sign to 


m 


yi A, (t), i.e. , when 
k=l 


m 
B(t) = -) aa A, (t) 


where Xr is a positive number, Substituting this value of B(t) into (5-32), we obtain 


m m 
a a2 S aliy=@A-mar’) Le 4,(0 | 2 (5-33) 
k=l = 


v 


53 


Since in this equality the expressions under the overbars are always positive, Quy isa 
minimum when 2A - m Aé is a maximum, i.e., when A = 1/m. Thus, to make the 
average energy of the signals AL (t) a minimum without changing the noise immunity, 
we must take aA 
Z 5-34) 
At) = Alt)-5 } ALC ( 
k=l 
It follows easily from this relation that 
m 
y Al(t) = 0 (5-35) 


k= 1 


We now study the system with m signals which has the minimum average energy for 
a given optimum noise immunity. If we assume that all the signals are equiprobable, and 
have the same probability of being correctly received, we can stipulate that sucha 
system consists of signals which are equidistant from one another. Take an arbitrary 


system of signals which are equidistant from one another, say the system 
B, (t), BA (t), +. y B_ (t) (5-36) 


for which 


2 2 
T (B,(t) - B, (t)) ae =, (5-37) 
independently of i and k (i # k). We now ascertain how much the energy of the Signals 
of the system can be decreased without changing its optimum noise immunity. We form 


the systern of signals 


m 


T m 
Bi(t) = B.(t) - = ee B, (t) (5-38) 


As already shown, this system has the same optimum noise immunity as the system of 


signals B. (t). The energy of the signal Bi (t) of this system is 


T Bie Qe <r [(m-1) By (t) - By(t)- ++. = BL (t)] 7 


— [(m-1)* BY (t) + BS (t) Been ee Be (t) - 2(m-1) B,(t) B,(t) 


- 2(m-1) B eee ~ 
(m-1) Ae) By() 2(m-1) B, (t) BL) 
(t) (t) (t) (t) +... B,(t) B_(t) 
+ 2Ba(t Balt + 2Bott B,(t) + + 2Bi(t Balt 
(t) B (t) +... 
+ 2B, t B, t) + + 2B, (t) Bw + 
+ 2B) By j 
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Moreover, bearing in mind that 


2B By = Be (t) + Be (t) - (B,(t) - B,,(t))” = BA(t) + Belt) - Bit (5-39) 


we obtain after some simplification 


2 m-1 ,2 2 
! = = 
ab BY (t) a 2m p = QR: (5-40) 
and similarly, 
2 m-1 g2 2 
T BJ (t) = Im 8 = Q: (5-41) 
Thus, all the signals Bi(t) have the same energy Onee 
To find the optimum noise immunity for this system, we form the system 
Bi(t) = B:(t) + C(t) (5-42) 


taking C(t) to be orthogonal to all the B.(t). Moreover, let us choose C(t) so that all 


the signals Bi(t) are mutually orthogonal, i.e., so that the equalities 


Brey BLE) = BY By + c*(t) = 0, oP ae 


are fulfilled. To do this, we must have 


GA Sox By Bw = 4 (Bit) - Bt (t))? -$ Br? (t) -5 Bi“ (t) 


But we have 


; ' 2. Ss 2 a2 
(Bit) = Br(ty) = (B,(t) - Bley) = B°/T (5-43) 


so that 


2 2 
T c(t) 7 Bb a ie = B* (5-44) 


EN? 2m 2m 


Thus, we can always choose C(t) so that the signals B: (t) are mutually orthogonal. For 


this system, the energy of the signals is 


BE «be he eee 2 ey F 
oo = BG) SP Bea Toe: Sore (5-45) 


Thus, the signals Bi'(t) have the same energy and are orthogonal, We have already found 
the optimum noise immunity for such signals. It is given by Eq. (5-12), where in this 
case we must substitute 

Q? - 3, = +p (5-46) 


The systems of signals Bi (t) and Bi\(t) have the same optimum noise immunity. 
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Thus, all systems of m signals which are equidistant from one another (have the same 


A) have the same optimum noise immunity. Systems of this type which have been obtained 
by the transformation (5-38) have the least possible average signal energy given by (5-41). 
These are the optimum systems (at least among the family of systems of equidistant 
signals), The optimum system of signals Bi'(t) given by (5-38) can be formed from an 
arbitrary equidistant system B.(t), for example, from an arbitrary system of orthogonal 
signals which have equal energies. The optimum system is somewhat better than the 
orthogonal system. Indeed, for the same optimum noise immunity, the signal energy in the 


optimum system has to be 


2 _ m-l 2 
°3.° mB (5-47) 


whereas in the orthogonal system it is 


2 2 2 
Q°= 8, = 6/2 (5-48) 


i.e. , m/(m-1) times larger. However, for large m, this difference is negligible. The 


system considered in Section 4-8 is the special case of the optimum system for m = 2. 


5-7 Approximate evaluation of optimum noise immunity 


The method of calculating optimum noise immunity discussed in Section 5-1 is often of 
little practical use, since in concrete problems the calculation of the probability that the 
inequalities (5-4) are satisfied presents great mathematical difficulties in many cases, 
Therefore, itis sometimes useful to have available a simple method of obtaining an 
approximate value of this probability. We now discuss this method. 

In order for an error to occur when the signal A, (t) is sent, it is necessary that one or 
more of the inequalities (5-4) fails to be satisfied, where we replace the index 1 by i, 
Alternatively, it is necessary that one or more of the reverse inequalities be satisfied; 


these reverse inequalities can be written after some manipulation as 


7) ‘ z 2, Pla) 
Oo Vet (A; (t) - A(t) 55 a (A;(t) - A,(t)" + 0% In PR) 
1. Sly 25 gag gy. eG (5-49) 


According to (2-47), the probability that the j'th of these inequalities is satisfied is equal to 


= > = 
Pi = PO, >a.) V(a,,) (5-50) 
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where 


2 
+ T (A.(t) - A, (t)) P(A.) 
a5 = td ee $ In Pay oo SE (5-51) 
Veo vost (A, (t) - A, (t))" 


As is well known from probability theory, the probability P that one or more of the 


events E)> E,, ieee En occurs always lies between the bounds 


m 
Be eas Sas y PE) 
k= 1 

where P(E.) is the probability of the event Ey, and P(E.) wax is the largest of the prob- 
abilities P(E), P(E,), Sb 5 P(E). Here P equals the quantity on theleft side of the 
inequality if the occurrence of one of the events necessarily implies that of the other events, 
and P equals the quantity on the right side of the inequality if the events E)> ede Ee 
are mutually exclusive. Using this, we can conclude that the probability P(A;) that one or 
more of the inequalities (5-49) is satisfied, or equivalently, that the transmitted signal 


A, (t) is incorrectly received, satisfies the inequality 


m 
imax fee iy ee (5-52) 
where P.. is defined by (5-50) and (P..) is the maximum value of P.. when the index 
i ij/max i 


j ranges from 1 tom. We note that in the inequality (5-52) the term Ps. should be omitted, 
since i # j in (5-49). This is accomplished automatically by setting P.. = 0. Multiplying 
the inequality (5-52) by the probability that the signal A, (t) is transmitted, which we 


designate by P(A,), and adding the resulting equations for i = 1, ... , m, we obtain 
m m m 
by) Pipnice POP SP ees P(A) Pi, (5-53) 
where 
m 
eo dy Pel) Py) (5-54) 


is the probability of error for the signals in question and for reception with the ideal 


receiver. 
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5-8 Example of the transmission of numerals by Morse code 


As an illustration of the method used in the preceding section, we determine the 
optimum noise immunity for the transmission of numerals with the use of Morse code. Here 
we shall assume that the amplitude of the signals is UL; that the length of a dotis To? 
that the length of a dash is 3%,» and that the space between a dot and a dash in one numeral 
is also T° We shall assume that the probability of transmission is the same for the 
various numerals. 

We denote the signal corresponding to the numeral 0 by A(t), to 1 by A,(t), to 2 by 
A,(t), -.. » to 9 by Ag(t). Then, as can easily be verified, if we subtract the value of the 
signal corresponding to the numeral j from the value of the signal corresponding to the 
numerali, and if we assume that the initial times of the signals coincide and that the 


frequency of the waveform is much greater than 1/%; we obtain 


T (A,(t) - A(t)? -v.U Yv (5-55) 


ij “o “%o 


where the iy are given in Table 5-1, Thus, according to Eq. (5-51), for this case we 


have 
@.. = a! 
qyY 
where 
(5-56) 
whence it follows that 
Pi =ViV¥ ij a!) for j # i (5-57) 
and P.. = 0, as already pointed out. On the basis of this data we can construct Table 5-2, 
where we have written 
Pie V(ia/n a!) (5-58) 
Then, keeping in mind that in this case P(A.) = P(A)) Bere = P(Ag) = 0.1 and applying 
Eq. (5-5), we obtain 
: é < < 
0 8P, +0 2P, < Pp = 1.8P, + 2.8P, + 1.6P, + 2.2P, + 0.6P, (5-59) 
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Table 5-1 
9 
: ee) ak ido ei 
jAi 
1 P, P, 7P4 + P,. 
2 P, 2P, + 3P, + 3P, + P, 
3 P, SP, + i . P, + P, 
4 P) 2P) + 4P, + P, + 2P, 
5 P, 2P) + 2P, + 2P, + 2P, + P. 
6 Py 3P, + 2P, + S + , 
7 Py 2P) + 5P, + Pi+ P, 
8 P, 3P, +3P,+2P,+ P, 
9 Py 3P, + ee ek 
0 P, Pi+ 2P, + Pt 2P4 + 3P, 


The bounds for PR: the probability of incorrect reception of a numeral, given by these in- 
equalities, are displayed as functions of a' in Figure 5-2. As is evident from the 
figure, the bounds for Pa lie quite close together. These curves allow us to determine 
the average percentage of incorrectly received numerals for the case of the ideal receiver, 
for a given signal to noise ratio, and for a given keying speed (on which the quantity ev, 
depends). If we carry out articulation experiments involving the reception by ear of 
signals representing numerals in the presence of noise of the fluctuation type, then the 
percentage of numerals which are incorrectly written down must be higher than 100P.., as 
,determined from Figure 5-2, By comparing these data, we can determine how close the 
noise immunity for aural reception is to the optimum noise immunity, i.e. , how much one 
can hope to increase the noise immunity of this kind of communication by improving 


reception, 
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Fig. 5-2. Bounds for the probability of 


error with the ideal receiver for numerals 
transmitted by Morse code; te is the 
length and US the amplitude of the 


elementary signal, 
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PART III 
TRANSMISSION OF SEPARATE PARAMETER VALUES 


CHAPTER 6 
GENERAL THEORY OF THE INFLUENCE OF NOISE 
ON THE TRANSMISSION OF SEPARATE PARAMETER VALUES 


6-1 General considerations 


In the preceding chapters we considered the transmission of discrete messages and 
signals. In this part we shall consider the transmission of a message which is a parameter 
which can take on any value within certain limits, and where the parameter is not trans- 
mitted continuously in time, but has its instantaneous values transmitted at certain time 
intervals, with a different signal being used for the transmission of each value. For example 
in telemetering we have to deal with transmission of this type. Im this case the signalis a 
function of time and of the transmitted parameter 2A, which is a constant for a given 


signal, We shall write such a signal as 


A(A, t) (6-1) 


If the noise W yt) is added to this signal, then the waveform acting on the receiver is 
? 


X(t) = Wary) + A(2, t) (6-2) 


Clearly, we would get the same waveform X(t) if another parameter, say |, was trans- 
mitted and if the noise took on a value Mi yt such that 
3 
W' (tt) + A(A',t) = X(t 6-3 
Nyy AIAG = Xt) (6-3) 


This is always possible, since as already remarked, normal fluctuation noise assumes any 
value with some probability, It follows from what has been said that in the presence of noise 
one can never determine with certainty from the received signal what value of the parameter 
XA was transmitted, 

In this chapter we shall determine the probability of the transmitted parameter having 
some value or other, when the received waveform is known. We shall find out what 
property the receiver should have in order to reproduce the most probable parameter value, 
given a received waveform, We shall call such a receiver ideal, Then we shall find the 
amount of error obtained when the ideal receiver is used to reproduce the parameter. We 
shall show that the mean square error has the smallest possible value for the ideal receiver, 
and we shall find this smallest value. This least possible error will depend on the signal 
form, and will characterize the optimum noise immunity for the given signal. The material 
considered in this part will also be used extensively later in studying the noise immunity 
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of telephonic communication with pulse modulation, 


6-2 Determination of the probability of the transmitted parameter 


Let the transmitted parameter A be a dimensionless quantity which can take on any 
value from -1 to +1 with the same probability. Clearly, if these conditions are not 
satisfied, then they can be satisfied by introducing a new parameter and suitably modifying 
the calculation, We assume that when the parameter which is to be transmitted lies in the 
range (k/m) < QA < (k+l)/m, where k = -m, -m+l,... , 0,... , m-l, we send 
instead the parameter Ax = (k/m). In transmission of this type we get errors which do 
not exceed 1/m, which is entirely permissible if m is chosen large enough. Obviously, 


under these conditions, the signal can have the 2m discrete values 


and we can apply to it the considerations of Chapter 3. Thus, if we assume that the 
received waveform is X(t), then, according to Eqs. (3-10) and (3-7), the probability that 
the transmitted parameter has the value Aw which means that the transmitted signal was 


A, (t), is equal to the quantity 


T (X(t) - AA,» 1)° 


exp] - = 
rar 7 m-l 2 
T_ (X(t) - A(£/m,t)) 
5 ep] - 2 BAe 
k =s-m Co 
as follows from the conditions 
P(A) sc = P(A.) = P(A) = oe = PIAL) 


which are satisfied by the signals under consideration, 
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It follows from this that the probability that when X(t) is received, the transmitted 


parameter A lies in the range 


aN < A < AS 


where A' = k'/m and Q"'5& k"/m, is equal to 


kn T (X(t) - A(k/m),t))* 
CSP Ya a ge 
KZ kk! Oo 
PA'<A<A')= 5 
mit T (X(t) - A(k/m,t)) 
. exp = ee ae 
k=-m o 


Now if we multiply the numerator and denominator of this fraction by AA =1/m, and let 


1/m approach zero, the sums approach integrals, and we obtain 


x: 5 
i segs |e 2) Sa) dn 
7 o 
PL(A' << D. < A") = i (6-4) 
J exp | - T (Xi) - Alay) an 
aT o 
Setting A" = A'+dA, we arrive at the expression 


o* 


=P (A')da 
+1 oS x 
exp | - 2~X(t) ~ AU, 1)) an 
hal com 
or (6-5) 
aaa 
PL(A) = K exp | - St (6-6) 


Oo 
where K, is a constant which depends on X(t) but not on A and t. 
It follows from what has been said that if we divide all the values of the transmitted 
parameter into intervals of the same length dA, then to the received waveform X(t) 


there corresponds most often the value of the transmitted parameter XA which lies in the 
interval 


Nye <A < Nem t 2A 
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where Dose is the value of the parameter X' for which the function P(A’) isa 
maximum. We shall call Ages the most probable value of the transmitted parameter A. 


It is clear from Eq, (6-6) that the quantity 


(X(t) - ACA, t))? (6-7) 


has its minimum value for XA = » eer If this function and its derivative with respect to 


A are continuous in A , then clearly Paid. must satisfy the equation 


ae 
aD (X(t) - A(A,t)) Fy eee 2 (Xt) - AQ LD) AAO? 4) = 0 


(6-8) 


where we have written 


ne eo 
Bae eae {% aah Stes (6-9) 


The receiver which, depending on the received waveform X(t), always reproduces Asm? 
the most probable value of the parameter, i,e., the value which minimizes the expression 


(6-7), will be called the ideal receiver. 


6-3 The function PLA) near the most probable value A sets 


We now find the quantity P(A) (introduced in Section 6-2) near its maximum, i.e., 
near the most probable value r SA ot The general form of this function is given by Eq. 


(6-6). If we assume that Ais near Asem? we can write 


A(A,t) = ADL ot + Al AL tA - AL) (6-10) 


Substituting this expression into Eq. (6-6) and taking into account the relation (6-8), we 


obtain 


-T(X(t) - Ag)? = TAS Ogee) (A + Asn) 


P(A) = K,. exp oe a 
2 (6-11) 
TANK. gt) 
= Ky exp | - Am (XA- Oe 


where K! is a constant which does not depend on 2X. Thus, the function Pd) obeys a 
x 
Gaussian curve in the region where Eq, (6-10) can be considered valid. If the noise 
intensity O- is sufficiently small, the exponent in Eqs, (6-6) and (6-11) becomes so large 
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in absolute value outside the region of validity of Eq, (6-10) that P(A) can be neglected 
outside of this region. In this case, we can regard the probability function Pid) as being 
given by a Gaussian curve everywhere, and the constant Ky. can be easily calculated from 


the condition 


+0 


j P(A) aA =1 (6-12) 


Substituting into this equation the value of P(A) from Eq. (6-11) and integrating, we find 


——— 
\/ Th at it 
Ki VT AAA (6-13) 


= vn o 


Therefore, for sufficiently small noise intensity, we can assume that 


VWray a an TAP) 


2 
exp | - ——————__ ()-/____) 
Vn oO = oe 


PLAA) = 


(6-14) 
It should be mentioned that in this case PAA) depends on the received waveform X(t) 
only to the extent that the quantity Nios depends on X(t). 

In these calculations, we assumed for simplicity that Eq. (6-11) is valid for all values 
of A lying between - «© and +o. However, this will not always be true, even for small 
Oo. In fact, PAA) must always vanish for A < -1 and A >+1, which means that Eqs. 
(6-13) and (6-14) can give a big error when A xern is near +1, Therefore, the results 
obtained in this section and in subsequent sections based on this one, require amplification 


in the case where Desi is near +1, 


6-4 Error and optimum noise immunity in the presence of low intensity noise 

Suppose that when the waveform X(t) arrives, the receiver, which is not necessarily 
ideal, reproduces a parameter AY which is a function of the waveform, We now determine 
the resulting mean square error. As already remarked in Section 6-2, P(A) dA is the 
probability that if X(t) is received, the transmitted parameter lies in the interval 
A, A+ aX. This is also the probability that the value of the parameter reproduced by the 
receiver has an error lying in the interval A-A,; A+tdAa - a, Therefore, in this 


case, the mean square error Sm is given by the expression 


+1 +1 +1 
{ (A-Ag? Pa) aA = | A’ PIN) ad 2a, | APA AA +R 
=). ~-1 -1 
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since 
+1 


| P(A) dA = 1. 


-1 


er. : 2 : . 

As is evident from this formula, é. varies with the choice of A, in accordance witha 
parabolic law, and has a minimum for some value = Axo" Differentiating ie. with 
respect to Ay and setting this derivative equal to zero, we obtain an equation for XA 


xO 
of the form 


+1 
2 
(Ad M/dA) = A..<* -2 J AP, (A) dA + 2A, = 0 
whence 
+1 
oo 7 if AP, (A) dA (6-15) 
-1 


or, what amounts to the same thing, Axo is the abcissa of the center of mass of the area 
under the curve P(A). We shall call A x0 the optimum value of the parameter A. 
If the waveform X(t) is received, then the minimum value of the mean square error, 


which is obtained if the receiver reproduces the value Aveo? is given by the expression 


+1 
So ) (A= Ag)” P(A) ad (6-16) 
=1 


It should be remarked that in the case where PAA) is a symmetric curve with a single 
maximum, then the abcissa of the center of mass of the curve obviously coincides with the 


abcissa of the maximum, which means that in this case 


Deges ercak (6-17) 


Thus, according to the result of the preceding section, we can assert that when the noise 
is sufficiently weak, in which case P(A) obeys a Gaussian distribution (which is 


symmetric), then Axo and Aig. are equal, and the ideal receiver gives the least mean 
square error. Using Eqs, (6-16) and (6-14), we can find that this error is 


2 
§ ee: aie (6-18) 
ee 2T AKA t) 


xm? 


This is the least possible error for sufficiently small &. It is obtained with the ideal 
receiver and obviously determines the optimum noise immunity in the presence of weak 
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noise. Here, and in what follows, we understand weak noise to be noise that has an 
sutedarey low enough to make the considerations of Section 6-3 valid. As is evident from 
Eq. (6-18), the optimum noise immunity for transmis sion of a parameter is proportional to 
the specific energy of the waveform AMA, yt) i.e. , of the derivative of the signal with 
respect to the transmitted parameter. 

Using Eq. (6-16), we can also determine the mean square error for large noise 
intensities. However, it is difficult to use this error to evaluate the optimum noise 
immunity. The point is that for large ©, the character of the function P(X) begins to 
depend on the received signal X(t), and therefore the quantity 6 am given by Eq. (6-16) 
also depends on X(t). In this case, in order to evaluate the noise immunity, we must also 
evaluate the probability of the various values of X(t), which leads to a series of mathematical 
difficulties. In Chapter 8 we shall return to the problem of the evaluation of the optimum 
noise immunity when the noise intensity is large. 

We now find the probability that, in the presence of weak noise, the ideal receiver 
reproduces the value of the transmitted parameter with an error exceeding ¢ in absolute 


value. Obviously, this probability is equal to 


Agent . +1 
Psl>e d= | Paar + J PCR) dA 
-1l Asma € 


Using Eq. (6-14), and keeping in mind the notation used in Eq. (2-47), we obtain 


oa 
2TAW (A, t) 
A = e J=2vi—&_) (6-19) 


P(\§ | >e) =2V 
dmm 


6-5 Second method of determining the error and optimum noise immunity in the presence 


of low intensity noise 


There is a second method of finding the size of the error for the case of transmission 
of a parameter in the presence of low intensity noise. Although this method gives a result 
which coincides with that already obtained, we shall examine it anyway, since this method 
is interesting in its own right, and since we shall use it later, albeit in a more complicated 
form, As before, let the signal A(,,t) represent some transmitted parameter X.. The 
noise Wa,v't) may or may not be added to the signal, with the result that a waveform X(t) 
acts upon the receiver; this waveform is A(A,t) if there is no noise, and A(J,t)+W wit) 


in the presence of noise. We represent the waveform by 


n 
X(t) = woe x,.C, (t) (6-20) 
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where the C,(t) are given orthonormal functions, Then X(t) is completely characterized 
by the values Kpreee yg Xe Depending on the received waveform X(t), the receiver repro- 
duces some value of the parameter A, a value which may or may not coincide with the 
transmitted value. We assume that to each waveform X(t) acting upon the receiver 
corresponds a specified value of the parameter, which is reproduced by the receiver. 


Clearly, for every receiver the reproduced parameter equals some function 


a he eee 5) (6-21) 


which characterizes its operation. 


Suppose the received waveform receives an increment 


n 
dX(t) = y, dx, C,.(t) (6-22) 
k= 1 


Obviously, in this case the parameter value reproduced by the receiver also receives an 


increment, equal to 
= oF 
dA = ry oe dx, = Lt) dx(ty (6-23) 
where we have designated 


n oF 
Lhe C, (t) (6-24) 
On ax, Tk 


as follows from (2-22). Suppose that the transmitted parameter is changed by dA and 
suppose that no noise is added to the signal; then the waveform arriving at the receiver 


changes by an amount 


dX(t) = AYA, t) dA (6-25) 
where 

ANA, t) = OAH) (6-26) 

yas 3A 


We assume that in the case where no noise is added to the signal, the receiver reproduces 
the transmitted parameter without error. Therefore, in this case the signal that is 


reproduced must also change by an amount dA. According to Eq. (6-23), we obtain 


dA = L(t) Ai(A,t) ANAt dA 


Thus, for a receiver which reproduces the transmitted parameter without error in the 


absence of noise, the relation 


Lit) ANA, ¢) AYA, = 1 (6-27) 


must be valid. 


69 


Now suppose that sufficiently weak noise Wa y(t) is added to the transmitted signal. 
) 


Then, due to the action of the noise the receiver waveform receives an increment 


dX(t) = Vis 


so that, according to Eq. (6-23), the parameter which is reproduced receives an increment 


o 2 
L(t) 08 (6-28) 


6 = dA =L(t) W (t) = 
for Vit 
The last equality follows from Eq. (2-60). Thus, when the receiver reproduces the para- 
meter value, the error obtained as a result of the addition of noise is a random variable 
which obeys a Gauss law. As follows from Eq. (2-50), the mean square error is given by 
the quantity 
2 
a TOR 6 2 
di, = Lt) Se (6-29) 
We now find what kind of receiver is needed to make the mean square error a 


minimum, Clearly, to do this we need to choose the receiver so as to make the quantity 


L“(t) a minimum, while satisfying the constraint (6-27). It is apparent that any function 


L(t) can always be represented as a sum of two terms 


L(t) = L(t) + L(t) (6-30) 
where the first term "coincides in direction" with the function AYA, t), i.e. 
Litt) = A'(A,t) (6-31) 


WHEE 0 is some constant, and the second term is orthogonal to this function, i.e., 


1 
Lo t Aq yt) = pe ae) ct) = 0 (6-32) 


Then 


Li) ANA, = LW ANA =p ANALY (6-33) 


which according to the condition (6-27) gives 


p ne (6-34) 


whence 
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AYA, t) 
L(t) = —A (6-35) 


ee 
A'y(A, t) 


As far as L(t) is concerned, it does not enter into the condition (6-27), and can take on 


any value. Moreover 


L(t) = Litt) + L(t) 


since 


It follows from this formula that, under the constraint (6-27), L(t) has its minimum value 


for L,(t) = 0. Thus we obtain the minimum value of the ervor if 
Al(X,t) 
Le. —A (6-36) 
2 
A'N(A, t) 


A 


so that, according to Eq, (6-28), this minimum error Sin equals 


f= ee Os: (6-37) 


Vix A'(X,t) 


A 


from which we obtain for the minimum value of the mean square error the expression 


2 Fs on 
2T AXA) 


which coincides with the formula (6-18) previously obtained for this quantity. 


6-6 Summary of Chapter 6 

The basic results obtained in Chapter 6 can be formulated as follows: Suppose the 
parameter A is transmitted using the signal A(A,t), which is a continuous function of X ; 
then the smallest mean square error produced by the addition of low intensity noise to the 
signal is obtained for the ideal receiver, which, when a waveform X(t) is received, repro- 


duces the value of the parameter X for which the quantity 


(X(t) - A(A,t))° (6-39) 
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has its minimum value. Moreover, when the parameter is reproduced, the probability of 
2 


at the mean square 


getting some value or other of the error obeys a Gauss law, and Jd 


value of the error is 


2 
re Snes o e Ate (6-40) 
2T AM(A,t) 
A 
where SA(A : 
L(ect\; oak (6-41) 
Al (Ast) aA 


This error is the least possible, and characterizes the optimum noise immunity for the 
signal A(),t) in the presence of low intensity noise, Thus, under these conditions, the 
optimum noise immunity is completely determined by the specific energy T AtZ(2, t), and 


the larger this energy, the larger the immunity. 


6-7 Geometric interpretation of the material of Chapter 6 


As we have already seen, a waveform can be represented by a radius vector, or, what 
amounts to the same thing, by a point of a multi-dimensional space. The discrete signals 
which we considered in the second part of this book could be represented by discrete points. 
The signals which we considered in this chapter can take on a continuous sequence of values, 
just like the parameter which they characterize. Therefore, the points which characterize 
the signal lie on a curve. We shall call this curve the signal curve, If a noise waveform is 
added to the signal waveform, then the resulting waveform is characterized by a new point 
which most of the time does not fall on the signal curve. 

As we have seen, if the waveform X(t) is received, the most probable value of the 
parameter is the one for. which the expression (6-7) is a minimum, i.e., the value corres- 
ponding to the point of the signal curve which is nearest to X(t). This is natural, since the 
shortest noise vectors are the most probable, which means that it is most likely that X(t) 


was formed by the addition of a noise vector to the nearest point of the signal curve. We saw 


that the larger the quantity ASA, t), the smaller the errors produced by the addition of 


noise. The quantity [ AN (A, t) yil2 


dd» characterizes the length of the element of arc 
described by the signal point on the signal curve, when the transmitted parameter is in- 
creased by dX. It is entirely natural that the larger this element of arc,the smaller the 
probability that such a displacement is produced by the action of noise. Thus, to increase 
the optimum noise immunity in the presence of weak noise, one should choose a communi- 
cation system in which the longest possible signal curve is obtained when the parameter is 


changed from -1 to #1. 
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CHAPTER 7 


THE OPTIMUM NOISE IMMUNITY OF VARIOUS SYSTEMS FOR TRANSMITTING SEPARATE 
PARAMETER VALUES IN THE PRESENCE OF LOW INTENSITY NOISE 


7-1 Amplitude modulation 


In this chapter we shall consider the noise immunity of some systems which are used 
to transmit separate parameter values in the presence of weak noise in the sense of Section 
6-3. First we shall investigate some modulation systems separately, and then we shall com- 
pare them. We consider first the case of amplitude modulation, where the signal can be 


written as 
A(A,t) = (1 +A) Bit) (7-1) 


where B(t) is some waveform or other, and 2 is a constant for a given signal and charac- 


terizes the value of the transmitted parameter. For this signal we have 
BAe) Ble} 


so that the minimum mean square error which characterizes the optimum noise immunity is 


given by the expression 


fii oe 
2T B~(t) 
where 
4T Bt) = T A“(1,t) = QF, (7-3) 


The quantity one is the maximum specific energy of the signal. 


Thus, in the case of amplitude modulation, the minimum mean square error which 
characterizes the optimum noise immunity is determined only by the signal energy and does 
not depend on the form of the signal, This result becomes quite apparent if we use the geo- 


metric interpretation of the type of modulation in question. In fact, for amplitude modulation, 
the signal curve is a straight line segment, one end of which is at the origin of coordinates, 
The longer this line, the greater the noise immunity, but at the same time the greater the 
length of the maximum radius vector of points on the line, and therefore the greater the 
signal energy corresponding to this radius vector. It is interesting to note that for amplitude 
modulation, any noise can be regarded as weak in the sense of Section 6-3, since in this case 
Eq. (6-10) is valid for all A. In this case, inaccuracy in the calculation of the mean square 


error comes about only because of the boundary effect mentioned at the end of Section 6-3. 
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7-2 Linear modulation 


The amplitude modulation just discussed is a special case of linear modulation, for 


which the signal is defined by the expression 


A(A,t) = AB(t) + BL(t) (7-4) 


where B(t) and B _(t) are any waveforms. It is easy to see that in this case the minimum 
° 

mean square error is also given by Eq. (7-2). However, with this modulation, it is possible 

to decrease the maximum specific energy of the signal, without changing the optimum noise 


immunity. For linear modulation, the signal curve is also a straight line segment, the 


length of which equals 2 V/ B (t). By properly choosing Bo (t), we may be able to shift the 
signal curve in such a way as to shorten the maximum radius vectors of the straight line 


segment, while keeping its length (and therefore the noise immunity) fixed. As is easily 
surmised, to do this we must take B A(t) = 0, Then the midpoint of our straight line seg- 
ment will fall at the origin of coordinates, and the maximum specific energy of the signal 
will have the smallest possible value, 


2) 2-9 Bey (7-5) 


2 

M 
ane : A 2 

so that the minimum mean square error will be expressed in terms of Qu as 


TE 
dmm ~ 5527 t=6) 
M 


Thus, in this case, we obtain a fourfold gain in power as compared with amplitude modulation 
However, the realization of this system entails technical difficulties, since in this case, the 
receiver must respond to the phase of the signal, which changes when A passes through zero. 
For the linear system of modulation, just as for amplitude modulation, any noise can be 
regarded as sufficiently weak. 

The application of non-linear modulation, for which the signal curve is not a straight 
line, allows one to significantly increase the noise immunity for weak noise without increas- 
ing the signal energy. The reason for this is that in this case the signal curve can be 
greatly lengthened by making it twisted, without thereby increasing the maximum distance 
between the points of the curve and theorigin of coordinates, i.e. , without increasing the 


maximum energy of the signal. 


7-3 General case of pulse time modulation 


We begin our investigation of non-linear modulation systems with the pulse time 
system, In this system, depending on the value of the transmitted quantity, the envelope of 
a high-frequency pulse can be shifted in time without changing its shape. For such 
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modulation, the equation of the signal can in general be written as 


TA 
° 
A(A,t) = U (« <> cos(w t + 9) (7-7) 
The quantity T, appearing here characterizes the maximum displacement of the pulse 


when A changes from -1 to +1, The receiver used with this modulation must somehow 
respond to the time shift of the receiver pulse, Usually with such a communication system, 
another signal is transmitted in order to establish a time origin at the receiver. However, 
we shall not be concerned here with these details, and shall assume that the time origin is 
known at the receiver, 

We now find the mean square error when low intensity noise is added to the signal, and 
when the reception is with an ideal receiver, For this purpose we use Eq. (6-38). For our 


signal we have 


TA 
ere ae nee ee 
AMA, t) = yl cos(wt + $) = - = cos(w. + 6) 
whence 
2 
=> t 
2. 
A(t) = ao 


We shall assume that the square of the term in rectangular brackets which appears in this 
term does not contain the frequency Zw, which is almost always the case, since the envel- 
ope UL ft) does not usually contain high-frequency components, Then, applying Eq. (2-26), 


we obtain 


2 2 
Tb, au (t) 
'S(A, t) as a —s— | 
since 
TA VT" 
t oa em aa 
and 


cos (ty, t +6) = 5 


It follows that when the ideal receiver is used, the value of the mean square error is given 


by the expression 
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40% 


ae 
oan: | (7-8) 
ouU__(t) | 
m 


eT [-E= 


As is evident from this formula, the error becomes smaller when v, or the specific energy 


of the waveform dU (t)/ dt is increased; the latter equals 


2 +T/2 2 
“ ou (t) 7 f 3 U_i(t) a5 
ot ot 
-T/2 


The error does not depend on the other signal parameters, 


7-4 Special case of pulse time modulation (optimum noise immunity) 


To obtain concrete results, we consider a special case of pulse time modulation. Let 


the transmitted signal be 


A(Q,t) = Un, (#- on. 


The envelope of this signal is represented by curve 1 in Figure 7-1. It has its maximum 
value at t = (TA )/2. The spectrum of the signal lies entirely in the band from 
(w, -{L)/2x to (a) + 1L)/2x 


We now find the minimum mean square error for this case. We have 


(t) aaa 3 Ax ae NY Me -MsinNt]? 
i t. t te i 
[ee] vw f E sind } at -u2 f [ Arcoete; Asiolt) i" 
S -T/2 


Letting the limits of integration go to + » , which can obviously be done, since T can be 


arbitrarily large, we obtain 


ou ft) 2 > 
2 soe Se (7-10) 


whence it follows by Eq. (7-8) that the minimum mean square error equals 
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oO 0) (3) 
m Le? iL 


‘T-2 (81g 
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2 
ee . a 5 (72) 
mm 
1 %, SLU, 


2. rs 
For ease of comparison with other systems, we shall express US in terms of the specific 


signal energy Q. According to Appendix A, the specific energy equals 


Qe = T AAA) = +2 ue (t) 
Thus 
2 aT/2 
2 Us sin“ SUt At hice ae ig (72) 
Bee cae Tee Zit *o 
Bu 


In this integration, to simplify the result we replaced the limits of integration by + >. 


Introducing this value in Eq. (7-11), we obtain 


52 - 6a" Gta 


mm x n 04 


° 
In this modulation system, all the points of the signal curve have the same distance 


va t) from the origin of coordinates, Hence the curve lies on some hypersphere, 
As is evident from Eg. (7-13), the noise immunity and therefore the length of the signal 
curve, increase when Tb, and {L are increased, while holding constant the specific energy, 


and consequently the radius of the hypersphere. 


7-5 Special case of pulse time modulation (noise immunity for the first method of detection) 


As is apparent from Eqs. (7-2), (7-6), and (7-13), the pulse time method of modulation 
discussed in the preceding section can provide great optimum noise immunity as compared 
with amplitude and linear modulation. However, for practical purposes, it is important to 
know how easy it is to realize this large optimum noise immunity. To clarify this question, 
we examine two concrete methods of receiving the signals considered in the preceding 
section, 

We assume that the receiving apparatus notes the instant of time to when the ampli- 
tude of the received signal assumes a certain value UL e.g. , suppose that at this instant 
a gas discharge tube flashes and that this is recorded on a moving light-sensitive film. 

Due to the action of noise at the receiver, this instant will be changed and cause an error, 
which we now find. We assume that there is an ideal filter in the receiver, which passes a 
band of frequencies from (wi- SL) /2 to (wW, +(L)/2n , i.e. , the pass band which con- 


tains the components of our signal. Then, obviously, we can consider only the components 
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of the noise which lie in this band. The sum of these components is the process Lar yt), 
“, 3 
given by Eq. (B-6) of Appendix B. Thus, the amplitude of the sum of the signal and noise 


waveforms can be expressed as 


Uy = Vitu_(t)+ 2 wi te) 7 + 2wi® (t) (7-14) 


m l,n 


If, because of the low intensity of the noise, we take ue (t) >> wi, then we can neglect 
) 


wi nit) and write 
? 


Us =U tt) + f2 ale (7-15) 


Let us see how much the action of the noise shifts the instant of time recorded by the 
receiver, i.e. , the instant to when the value of the amplitude of the received waveforms 
assumes the value UL: In Figure 7-1, curve 1 represents the dependence of UL on t, and 
curve 2 the dependence of the sum amplitude Us on t. According to Eq. (7-15), the 
vertical distance between these curves is the quantity -V2 Watt) It can be seen from the 
figure how much the time instant to is shifted by the action of the noise. This shift gives an 


error in the determination of A equal to 


t' -t 
e) 


eer a ae (7-16) 


fe) 


since ty is shifted by %,/2 when A is changed by unity. Here we have denoted by i 
the instant of time when the amplitude of the signal takes on the value UL. We assume that 
the size of the error dS is small enough so that for the time an - ae the quantity Waa) 

can be regarded as constant and the segment of the curve UL t) can be regarded as recti- 


linear. Then the ratio between S and wy alto) can be found from the figure, and is clearly 
) 


(7-17) 
ot us - ts 


ou, (t) Na 
t=t' 
fe) 


In view of Appendices B and C, we obtain from this 


2(t' - t_) 
Sh i 
% [| 


t=t! 
° 


where 6 is a normal random variable. As we see from this formula, as in the case of the 


ideal receiver, the error 6 obeys a Gaussian law, The mean square error equals 
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i Stee asic’ (7-19) 
E = =; 
m 
270 : t= t! 
fo) 
From Eq. (7-9), we obtain 
oU__(t) : 
m e x cos x - sin x 7-20 
: t=t! - oe A 


where 
GA 
ao 


: : 2 
Giving x various values, we can use Eqs. (7-19) and (7-20) to determine the quantity 3 rn? 


which can be written as 


2 2 
2 1206 607 
§ = eee (7-21) 
a m va tv, Ato fL Q 


where Ay is a function of x, and oF is defined by Eq. (7-12), Using the formula 
Sy. Ss za 2) 


we can find the dependence of x on UL/U; and consequently the dependence of c/a on 
UL/U,; which is given by the appropriate curve in Figure 7-2. Comparing Eqs. (7-13) and 
(7-21), we see that since 2 is always less than one, the noise immunity for the method 
of reception discussed in this section is less than the optimum noise immunity; moreover 
2, is the efficiency coefficient (introduced in Section 4~2) which shows how much the 
signal energy (strength) can be reduced with the ideal receiver, while obtaining the same 
noise immunity, i.e., the same So; as with the given means of reception, As can be 


seen from Figure 7-2, al has its maximum value of 0,58 at UL/U, = 0.41, 


7-6 Special case of pulse time modulation (noise immunity for the second method of 


detection) 


We now try to decrease the mean square error as compared with the value we obtained 
for the first method of detection, described in the preceding section. Such a decrease is 
possible, since the error obtained was larger than that given by the ideal receiver. To 
achieve this, we use a receiver which reads not only the instant of time t, when the ampli- 


tude of the received waveform crosses the level UL from below, but also the instant ty 
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when the received waveform crosses the level UL from above; we take the value of the 
transmitted parameter to be the mean value 

t + ty 

2 (7-23) 


As a result of the addition of noise, we again obtain an error both in the reading of the 
instant t_ and in the reading of the instant ty. We denote these errors by Sg. and Sy: 
re) 


respectively. Clearly, by Eqs. (7-16) and (7-17) we have 


a(t) - t,) 2NV2 WT nftg) 


OES axe 7k (7-24) 
2 VC, 3U_, (t) 


ot 


Fog sec ae Seam 3 eee (7-25) 
d bo au (t) 
Lae 
at t= tl 


where ts is the instant of time when UL (t) passes through the value UL from below, and 


th is the instant of time when UL (t) passes through the value Us from above. Since 


UL ft) is a symmetric function, we have 


oU__(t) oU_ (t) 
perenne) AN ee 4 eee (7-26) 
Bt tt o t =ti 
1 ° 


Therefore, the error obtained in reading the mean value equals 


2 Sy iJ 5, 2 CW alt a Wi nly) 


{2 


(7-27) 


We now find the random variable Ww) nite? - wy nitty): Here we cannot use directly 
) d 
the result of Section 2-5, since the random variables Ww nfts) and wi nt) are dependent. 
? 
According to Eqs. (2-54) and (2-74), we can write ; 


2n 
Y (lg (th) ~ Ip (tr)] eg 


" as uw C 
ae Wa) 


o 
V¥2T Q=1 


t ' 2 
[ip (ty) - Piel” 8 
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Now we take two terms of this sum and consider Eq. (2-14). We obtain 


; 2 2. 2a 3 
(15, _ (th) 4 1; (ti) J es (1, (tt) te 1, (tt) J = 4[1-cos re i (tt t1)) 
whence 


n 
" yoo " ' = 2ni 1 
WH tts) WH tty) V20 V va l-cos = (t! - t1)) z 9 


Bearing in mind that n = ({L/2x)T, letting T approach @ , and introducing the notation 
x = i/T, dx = 1/T, we arrive at the expression 


f/x 


" ty. " 'y = 
we) WT (th) Vie : {1 - cos 2n (th - ti )x] dx @ 


(7-28) 
Substituting this value in (7-27), we obtain 


sinft(t' - t!) 
al 1 
ey ae |S 


(7-29) 
oU__(t) 
m 
Fo | ot | t=t! 


As we See, in this case also the error is a random variable which obeys a Gaussian law, 


d = 


It follows from the formula just obtained that the mean square error for the given means 
of reception is equal to 


jee [: ; sin/L(t! - a 2 


“REL = ti | 
Sin * a a 
7 du _(t) 
#%6 ot i t! 


Toa 
Giving UL/US various values, and using Eq. (7-2), we can find the quantity x=/L fs ="), 
which by the symmetry of UL ft) equals sit: - th). Then we can use Eqs. (7-20) and 
(7-30) to find the quantity hee obtaining 


2 2 
St ce = ~ °F, (7-31) 
2. 5 fLUS 2, Vote Q 
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where the quantity Xo» which is a function of the ratio UL/US; is given by curve 2 in 
Figure 7-2. A comparison of this with Eq. (7-13) shows that 2> is the efficiency coeffic- 
ient for the given means of reception. As is evident from Figure 7-2, 72> is greater than 
2 and is near unity, approaching unity when UL/US —) 1. Thus, this kind of reception 
is more immune to noise than the kind analyzed in the preceding section, and for all 
practical purposes achieves the optimum noise immunity. We obtain the optimum noise 
immunity if UL/U, = 1, i.e. , if we take the readings near the peak of the pulse, or, what 
amounts to the same thing, if we determine the transmitted parameter by the position of the 
maximum amplitude of the pulse, It is clear that in the presence of weak noise, one cannot 
achieve a better means of reception than the one discussed in this section. The reason for 
the increase of noise immunity is clearly contained in the fact that in most cases (especially 
when UL/U,S is near one), the noise which is added to the signal either simultaneously 
raises or simultaneously lowers both sides of the envelope of the signal pulse, so that the 
mean value of the quantities ty and ty changes less due to the action of the noise than 


either of these quantities separately. 


7-7 Frequency modulation (general case) 


We now consider the noise immunity of a system which transmits continuous values 


(of parameters) by the use of frequency modulation. In this case the signal can be written as 


A(A,t) = Up (t) cos Uw, + MLA)t + 051 (7-32) 


In order to find the minimum mean square error, we apply to this signal the basic formula 
(6-40). We find 


AMMA, t) = - U_(t)/Lt sin E(w, +ILA)t + 0,1 


and 


AYA. = (2 unin? sin“[ (w, + MLA + 8 


m 


2 2 ‘ : 
If we assume that the waveform UL (tye does not contain the frequencies 2(W, +LLA), then 


according to Eq. (2-26), we obtain 


Atg(Ast) = 31 fury 


since we have 


sin ((w, +L AE+ O01 =5 


for sufficiently large T. Substituting this value in Eq. (6-40), we obtain 


84 


ne = ee (7-33) 
an TUS (t)t” 


Thus, the larger SL and the larger the specific energy 


+T/2 
t ue (tt” = J tu= (t) dt (7-34) 
-T/2 


of the waveform tu (t), the smaller the error. Itis apparent from Eq, (7 -34) that this 
specific energy is proportional to the moment of inertia about the line t = 0 of the area 
under the curve us (t). According to (A-2), the specific energy of the signal under con- 


sideration is 


+T/2 
Q° = T AA, =5 { u(t) dt (7-35) 
-T/2 


provided that the oscillation ue (t) does not contain the frequencies 2(4) +SLd), and pro- 
vided that T is sufficiently large, Thus, this energy is proportional simply to the area 
under the curve ue (t). 

If we wish to increase the optimum noise immunity without increasing the signal 
energy, we have to increase the moment of inertia about the line t =0 of the area under 
the curve ue (t) without increasing the area, Clearly, this can be done by increasing the 
ordinates of the curve in parts which are far from the origin, and decreasing them in parts 
which are near the origin. By simply time-shifting the envelope of the signal further from 
the origin, we can also increase the moment of inertia, and therefore the noise immunity, 
without increasing the signal energy. This last fact may seem strange, but is easily ex- 
plained. In fact, when t =0, the argument of the cosine in Eq. (7-32), and therefore the 
expression itself, does not change when A is changed; the larger t, the greater the change, 
which must lead to an enhancement of noise immunity, Therefore, shifting the envelope 
must actually lead to an increase in noise immunity. The significance of this shift can also 
be explained by the following mathematical transformation. If we time-shift the envelope 


of the signal (7-32) by an amount ti» we obtain 
U it = t5) cos Maw, +f A)t + 6) = Ut!) cos Fw, +fLAM(t +t.) + 0] 


= UL (tt) cost (w, +SLAIt! + wt, +LAt, +0) (7-36) 
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where we denote t-t_ by t'. We see from this last expression that shifting the envelope 
o . 
is equivalent to making the initial phase of the cosine change with A ,» which changes the 


noise immunity as well. We shall consider such a system in Section 7-9, 


7-8 Frequency modulation (special case) 
In this section we consider the special case of frequency modulation in which the signal 


is a section of a sine wave of constant amplitude, i.e. 


A(A,t) = Uj cost(w, +fLAre+ oO), for = -t/2<t<%,/2 
(7-37) 
A(X, t) = 0, for t <- T,/2 and t > 12 
In this case the envelope can be expressed as 
= f - 2<t< 2 
Ut) UL; oF. %,!/ by Ss v,! (7-38) 
Ut) = 0, for t<- e,/2 and t > Tle 
Therefore, according to the general formulas (7-33) and (7-35), we obtain 
ee 
Q = Zz %, (7-39) 


Hin oo ee | ee (7-40) 


Since for frequency modulation, the specific energy of the signal does not change when 
the transmitted parameter A is changed, the signal curve lies on a hypershpere, just as for 
pulse time modulation, Comparing Eqs, (7-13) and (7-40) for pulse time modulation and 
frequency modulation, we see that the size of the minimum mean square error is given by 
the same expressions for both kinds of modulation. However, the quantities L\ and %, 
entering into these expressions have a different meaning. In Eq. (7-13), (1 designates 
half the bandwidth occupied by the signal; in Eq. (7-40), wal designates half the maximum 
frequency change of the signal. However, the frequency band required for the transmission 
of signals by frequency modulation can be regarded as approximately equal to the maximum 
frequency change of the signal, Therefore, in both formulas, wal designates half the fre- 
quency band needed to transmit the signals, In Eq. (7-13), b, designates the maximum 


time-shift of the signal pulse; in Eq. (7-40), b, designates the signal length. However, 
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the time needed to transmit the signal by pulse time modulation can be regarded as approx- 
imately equal to the maximum time displacement of the pulse, Therefore, in both formulas, 
bo designates the time required to transmit the signals. Thus, we obtain the same opti- 
mum noise immunity for the transmission of signals by pulse time modulation and frequency 
modulation, provided that they have the same duration, the same frequency band, and the 
same energy. 

Comparing these two forms of modulation with amplitude modulation (Section 7-1), we 
see that they afford greater noise immunity in the case where ee or > 1, The three 
kinds of modulation considered as examples are far from exhausting the very large variety 
of possible schemes, We saw that increasing the noise immunity for amplitude modulation 
required an increase of signal energy. With frequency modulation and pulse time modulation, 
we were able to increase the noise immunity in the presence of weak noise without increas- 
ing the signal energy; rather, it was necessary to increase the time or bandwidth occupied 
by the signal, In the next section, we shall consider ways of increasing noise immunity in 
the presence of weak noise which do not require that we either increase the signal energy 


or that we increase the time or bandwidth occupied by the signal, 


7-9 Raising the noise immunity without increasing the energy, length, or bandwidth of the 
signal 
In this section we consider systems where it is possible in theory to increase indefin- 
itely the optimum noise immunity in the presence of noise with sufficiently low intensity, 
without thereby increasing the energy of the signals or increasing the time or bandwidth 


occupied by them, Let the transmitted signal be defined by the following expression 


A(A,t) = u. cos Kw +fLA)t+9+aAJ), for - El2sts 12 


(7-41) 
A(A,t) = 0, for t< oe and t > t,/2 
Thus, this signal differs from the one discussed earlier in connection with frequency modu- 


lation in that its phase also changes in accordance with the transmitted parameter, For this 


signal we obtain 


A(X, t) = - uj(fLe + a) sin [(w, 4#(LAjt+@+aA], for - Cl25 tS Bl2 


whence 
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+4 [2 


T AIA.) =U, (Dt + a)? sin? (w, +A + 9+ ard at 
-%,/2 
+y/2 +%/2 
= 8 if A (Mt+ a’ de - ue a 5 Ut +a) cos 20 (ay, + NAt+ H+ adjat 
-%,/2 -%,,/2 


The second integral goes to zero as ww, is increased, and can therefore be neglected for 


sufficiently large w.- Thus we obtain 


eee 2 


U 
i AL (Ast) ast UVa + 12%, a) 


from which it follows that the minimum mean square error characterizing the optimum noise 


immunity for the signals in question is given by 


2 2 
Soe = cee He (7-42) 
USL” % + 12%, a) sa a (1+ —— :-) 


As can be seen from this formula, the error can be made arbitrarily small by increasing a. 
At the same time, changing the value of a does not change the energy, bandwidth, or 
duration of the signal. An analogous result is obtained in the case of pulse time modulation, 
discussed in Section 7-3, if we change the phase of the high frequency oscillation in pro- 
portion to the transmitted parameter, 

In practice, it is quite difficult to realize the optimum noise immunity of these systems, 
since to do so we require a receiver which responds to the initial phase of the high frequency 
signal oscillation. However, it is possible to propose modulation systems for which it 
would be easier to realize great noise immunity in practice. An example of such a 


system with the signal 


A(4,t) US [l + cos( {Lt + ad) J cos(4@) +AMUNt, for -%, (2 <t< w,/2 
(7-43) 


A(A,t) 


0, for t<-%/2 and t * e,/2 


This signal has an advantage over the signals considered above in that it undergoes a change 
in phase of the low frequency oscillation rather than of the high frequency oscillation. This 
phase is changed less when the signal is propagated, and is more easily detected by the 


receiver. 
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The examples considered are far from exhausting all possible versions of modulation 
systems for which the noise immunity in the presence of weak noise can be made arbitrarily 
large. For example, such modulation systems can be constructed by the following general 
principle, Some signal parameter, e.g., a phase, has to change in accordance with the 
transmitted parameter » and an arbitrary change of this signal parameter must not in- 
crease the time or frequency space allotted to the signal, nor increase its energy. Thus, 
this parameter can be changed by an arbitrarily large amount, which thereby makes the 
signal curve arbitrarily long and arbitrarily increases the noise immunity in the presence of 
weak noise, However, the variation of this parameter alone is usually not sufficient, since 
it produces periodic changes in the signal, so that the same signal will correspond to differ- 
ent values of the parameter, To remove this multiple-valuedness, it is necessary to 
simultaneously vary some other parameter as well, e.g., the frequency of the oscillation, 
its amplitude, the location of the signal pulse in time, etc. This change must be confined 

within certain limits, since it usually produces a change in the signal energy, or a change 
in the time or frequency space allotted to the signal. 

The defect of the systems considered in this section is revealed if we study their 
noise immunity inthe presence of noise of high intensity, a topic to which the next chapter 
is devoted, It is found that the larger we make the noise immunity by the methods presented 
in this section, the lower the noise intensity at which the boundary between "strong" and 
"weak" noise occurs, and the formulas which we have derived are not valid for "strong" 
noise. In the limit, the methods presented here allow one to reduce to zero the error 
resulting from the action of ''weak" noise, but at the same time, ''weak'' noise comes to 
Mean noise with an intensity which is itself equal to zero’*, Thus, we cannot succeed in 
completely nullifying the action of noise by these methods, as might otherwise be expected; 
we can only obtain a reduction of its effect. This reduction is worthwhile for communication 
in the presence of noise with sufficiently low intensity, when it is necessary to have very 


few errors, 
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CHAPTER 8 


NOISE IMMUNITY FOR TRANSMISSION OF SEPARATE PARAMETER VALUES 
IN THE PRESENCE OF STRONG NOISE 


8-1 Derivation of the general formulas for evaluating the effect of high intensity noise 


In this chapter we evaluate the optimum noise immunity for transmission of parameters 


in the presence of high intensity noise. We denote by ae (A>A,) the probability that when 
‘a 


the parameter value A, is transmitted, the receiver, as a result of the addition of noise 


to the signal, reproduces a parameter A satisfying the condition A > A,; by Py (> <A,) 
a 


we denote the probability that when the parameter value A, is transmitted, the receiver, 
as a result of the addition of noise to the signal, reproduces a parameter value satisfying 
the condition A < A: Obviously, these probabilities depend on both the method of trans- 
mission, i.e. , on A(A,t), and on the method of reception. With this notation, the prob- 
ability that the error § exceeds ¢€ in absolute value, if the parameter x, was trans- 


mitted, equals 


Py (a? At a+ PA IAS Cr) 


We shall assume that the transmitted parameter A, can take on any value in the range -1, 


+l, with equal probability, Then the probability that A, satisfies the inequality 
Az< ALS Ag + AQ 
and that at the same time \d | > e€ , is equal to 
dr, 
P, > & —— 
[ a,'4 Az +e) + Pa <2, -ell = 


Hence, the probability that the error exceeds ¢€ in absolute value, when a parameter value 


a, (not known in advance) is transmitted, equals 


+ 
ry 


da, 
[Py Ar hg +te)+ PAA < de - ¢)) ae as 


P(|d| > e) 


' 
— 


+ 
_ 


adr fe dr 
= 2 
= Py (A? A» +e Jae it i) PAAWA <A, se) - 


i] 
— 


Clearly, the value of the integrals is not changed if we substitute the quantity A = »X +e 
° 


into the first integral, and the quantity A. = AS -e€ into the second, making correspond- 
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ing changes in the limits of integration. Then we obtain 


P(|§|> «) cia (A 4) 2A age De eee 
€ = > + 
Fon ee eeige” <o ae ay o 2 
l-e ar. 
> ao Ba uselA a Ag) Ea cl <A.) <= 
-(l- € 


since the expressions under the integrals are always positive. 

Let us digress a bit to calculate the quantity in rectangular brackets. Let A(t) 
= A(A,- e€,t) and A,(t) = A(X, + €,t) be two discrete signals, such as were discussed 
in Chapter 4. Let the receiver under consideration, which serves to determine the para- 
meter 2% , be used to receive these signals, We shall say that the first signal A(t) 
= A(X, - €,t) was sent if the receiver reproduces a A < a, and that the second signal 
was sent if the receiver reproduces a A > Ao: Then the probability of error for these 
signals and the given receiver is 


z IP, (A >A) +P 


a (A<A,)] 


Aot € 
is we assume that the.a priori probability of transmitting either signal is the same, How- 
ever, this probability of error cannot be less than the probability of error (given by Eq. 


(4-8)) which determines the optimum noise immunity for the signals in question, i.e. 


5 IP, _ (A >A) + 


e (A<2)1 > V(a)) 


xt € 


where V(a) is defined by Eq. (2-47); ay is defined by Eq. (4-4) and in this case equals 


sys /=, [A(A,+e 51) - A(A,- es0))? 


[AIA te - AA: ev) at 


(8-1) 


20 -T/2 


From this we obtain the universal formula 


1-€ 
P({g| > «) > J V(a,) dA, (8-2) 
-(1-€) 
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for calculating the probability of errors greater than ¢. In many cases os does not 


depend on Ao: If this is the case, the quantity under the integral is constant, and we 


obtain 


P(|S] >< )>2(l-e) V(e)) (8-3) 


It follows from these equations that the smaller the distance 


V | A(A, +¢€,t)- A(A, -e ,t) \¢ 


between the points of the signal curve corresponding to parameter values which are separated 
from one another by the amount 2e, the larger the probability of obtaining an error J 


exceeding e. 


8-2 Comparison of the formulas for weak and strong noise 


We now compare the result obtained in the preceding section with the result obtained 
in Chapter 6 for the case of weak noise; there we derived Eq. (6-19), which gives the prob- 
ability that the error d is greater than e¢ , for the ideal receiver in the presence of weak 
noise, This formula is valid for a given most probable value Ages If we assume that all 
Xn are equally probable, then when Ages is not known in advance, we obtain the follow- 
ing expression for the probability in question: 

+1 
PIS | >) = J V(a) day (8-4) 


+/ 2 
2T AN (ZL , t) 
Qa See et A € (8-5) 


o 


where 


Let us compare this result with the result given by Eq. (8-2), which is universal and is 


Suitable both for strong and weak noise, For small ¢ , we can take 


A(A, + €,t) - ACA, - et) = AMA,» 2 € 


Substituting this value in Eq. (8-1), we obtain 


Vet aw(yr st) 
OG oe A or € 


1 o 


(8-6) 
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This quantity is to be substituted in Eq. (8-2), which gives a lower bound for the probability 
of error, From these formulas, we see that a = Oy» which means that the right sides of 
Eqs. (8-2) and (8-4) differ only by their limits of integration, a difference which goes to 
zero as ¢€ ->0, It follows from these expressions that if the inequality (8-2) is changed to 
an equality, then it gives the value of the probability of the small errors produced by the 


ideal receiver in the presence of weak noise, 


8-3 Pulse time modulation 


For amplitude and other linear modulation, the formulas obtained in Chapter 7 are 
valid for noise of arbitrary intensity, and therefore there is no point in investigating these 
kinds of modulation using the methods of Section 8-1, The situation is different in the case 
of pulse time modulation, For this kind of modulation, according to Eqs. (2-26), (7-9) and 
(8-1), we obtain 


2 ve sinh [e-e 0+ ef sinft eae || ? 
oy 2 yt 


20° A [ eBags ale oes a 


-e 


v t 
= sintL | t - = (A+ €) sin tL | t - z= o> =) a 
mc _ t 
40° -T/2 Lye xo (A+ ft 
= ear LAS e) t war aah e) 


Changing the limits of integration in this integral to -* and + , introducing the value 


of the specific signal energy Q* given by Eq. (7-12), and integrating, we obtain 


Pie sintl@ « 2 sinfl % « 
a? = —_ 1-2 |= 8 Poe OY (ees) 
20° (L SIL, € oy ALT «€ 


AS we see, Oy does not depend on A? which means that we can use Eq. (8-3) to calcu- 
late the probability of error. 

The curve a, b, c, d, e in Figure 8-1 gives the dependence of the quantity in paren- 
theses in Eq. (8-8) on the parameter At « , which is plotted as the abscissa, This 
quantity determines a) and 2V(a ) for a given value of Q/o. In the figure there are 
five scales along the axis of ordinates, from which the value 2V(a 1) can be found directly 
for the values Q/o = 1, 2, 3, 4, 6. Since for small € the quantity P(|$| > ¢ ) must be 
larger than 2V( a, ), and moreover must be a monotonically decreasing function of e, then 


for ¢ <1, the es representing the dependence of P( \S|> ¢) on ¢€ must lie above the 
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Fig. 8-1. The curve a, b', c, d', e is the 
lower bound for the probability of an error 
greater than ¢ for pulse time and frequency 
modulation and strong noise, for different 
Q/o (several scales along ordinate axis); 
Q is the specific signal energy; (L/n is 
the bandwidth and Tv, the time occupied by 
the signal; the curve a" is the probability 
that the error exceeds e , obtained from 


the weak noise formula. 
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curve a, b', c, d', e, obtained from the curve a, b, c, d, e by filling in its valleys. 

This must be the case for any means of reception, including real reception. Thus, the 
value of the probability P( ld |> ¢) which characterizes the ideal receiver must lie above 
the curve a, b', c, d', e. In the case of weak noise, for small values of the quantity 

LT, € and for ideal reception, we can determine P(|§ | >e), using Eqs. (6-19) and (8-5). 
This quantity is represented in Figure 8-1 by the curve a". It is apparent from an examin- 
ation of the figure, that for ALT, < 2.7, the curves a, b', c, d', e and a" are quite 
close together. However, for Ay®€ > 2.7, we obtain a drastic divergence between them, 
with the curve a'’ going below the curve a, b', c, d', e, which is impossible, as remarked, 
It follows from this that for NT, e > 2.7, the formula for weak noise and small errors is 
completely inapplicable, 

We now clarify these results, For the given means of communication and for the 
methods of reception described in Sections 7-5 and 7-6, small errors are caused by weak 
noise, which produces a displacement of the sides of the pulse. The probability of this type 
of error falls off sharply as the error is increased, Large errors are obtained when the 
noise waveform exceeds the threshold voltage U_. Itis clear that this can happen with 
almost equal probability at any time. Therefore, the probability of large errors does not 
fall off much when the error is increased. This property, which is easy to explain for the 
method of reception in question, is (as shown by Figure 8-1) a necessary feature of the 
given means of communication, regardless of which means of detection we use, The large 
errors, for which the formulas derived in Chapter 6 for weak noise are not valid, will be 
called anomalous. As we see from Figure 8-1, anomalous errors must begin at least from 
the value ¢ = 2.7/SL %, on. For example, it is clear from the figure that for Q/o = 2, 
the probability that an anomalous error occurs, must be greater than 6 x er This means 
that in more than 6 percent of the cases, on the average, anomalous errors occur for the 
given value of Q/o-. In general, the probability of occurrence of anomalous errors can be 
found using the fact that they begin when SL, ¢ > 2.7. Thus, according to (8-8), these 


errors begin for 


which means that their probability is 


P(S$ anomalous) > 2v(2) (8-9) 
te o 


For low intensity noise, the probability of anomalous errors is very small, so that they 


need not be considered and the weak noise theory can be applied. 
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8-4 Frequency modulation 
We now apply the results obtained in this chapter to the case of frequency modulation, 
considered in Section 7-8. We have a signal given by Eq. (7-37). Applying Eq. (8-1) to 


this signal, and taking (2-26) into account, we obtain 


2 
TU ——— .h_OOOOOOT 
af = aa fcos[ (wot LA, +fL et + 6) - cos [(w,+ La, -fLe)t+ $)% 
2 2 +%u_/2 
2TU Se U ° 
= — sin“fL et sin” [(w, + SLA t+ 6) = — J sin2fL et dt 
Oo oO -t,[2 


Doing this integral, and introducing the value of the specific signal energy given by Eq. 
(7-39), we have 


(8-10) 


a. =) 


2 Q2 aa sin tLe ) 
1 o-* DES, € 


Comparing this formula with the formula (8-8) for a% in the case of pulse time modulation, 
we see that they are identical, Therefore, all the results obtained for pulse time modulation 


are applicable to this case also, 


8-5 The system for raising the noise immunity without increasing the energy, length or 
bandwidth of the signal 


In this section, we shall evaluate the noise immunity in the presence of strong noise 
of the system which we discussed in Section 7-9, This system allowed us to make the noise 
immunity arbitrarily large, provided that the noise was sufficiently weak, In this case, 
the signal is given by Eq. (7-41), and by a calculation completely analogous to that of the 


preceding section, we obtain 
a* s af ie Pe. Solr. cacstae (8-11) 
1 € 


Figure 8-2 shows the curves giving the dependence of the quantity in parentheses in Eq. 
(8-11) on the value of the parameter Ne, e , for three values of a, i.e., a = 0 for 
curve l, a = LT, for curve 2, and a = 21. %, for curve 3. In accordance with the 
considerations presented in Section 8-3, the valleys of these curves have been filled in. For 
a given value of Q/o-, the quantity in parentheses in Eq. (8-11) determines the values of a 


and 2V( a,). The value of 2V(a,) can be read off at once by using the scales along the axis 
of ordinates in Figure 8-2, 
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Fig. 8-2. Lower bound for the probability of 


an error greater than ¢ for the signals given 
by Eq. (7-41) for various Q/O-, Curve lis 
for @ = 0, 2 for a=1LT,, 3 for a= 2%; 
Q is the specific energy, Ge the duration, 
and +LL/2x the maximum frequency devi- 


ation of the signal. 
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As we have shown, the probability P(|$| >«¢) has to be greater than or equal to the 
value of 2V( a1); if we neglect the effect of the factor l- € in Eq. (8-3), and moreover 
equals the value of 2V(a,) for small values of the error e€, for weak noise, and for recep- 
tion with the ideal receiver. It is apparent from the curves shown that the noise immunity 
for small € , i.e., for small errors, increases as a increases, Thus, under these cir- 
cumstances, the curves do not restrict the validity of the results obtained in Section 7-9 for 
this modulation system, On the other hand, it follows from Figure 8-2 that the larger a, 
the smaller the values of € at which anomalous errors occur, and the larger the probability 
of such errors. This proves the statement made at the end of Section 7-9 concerning the 
defects of this and similar modulation systems. However, it is apparent from the curves 
that if the value of a is not made too large, then the use of such a modulation system can 
be worthwhile, In fact, comparing the curves of Figure 8-2 for a = Oand a= fl T» we 
see that for small e€ the value of the quantity in parentheses in Eq. (8-11), and therefore 
. is ten times larger in the second case than in the first, which greatly in- 


1 
creases the noise immunity for weak noise, However, in the second case, anomalous 


the quantity a 


errors begin to occur (as shown by the figure) when the quantity in parentheses has the 
value 0,78, which is before the value of 2.7 at which they begin to occur in the first case. 
Therefore, in the second case, the probability of occurrence of anomalous errors is greater 
than 2V(-/0. 78 Q/o-), while in the first case it is greater than 2V(~/2.7 Q/o). If we go 
to the case a = 2.0, , then, for small € , the noise immunity increases further, but at 
the same time the probability of anomalous errors increases appreciably, and is in this 


case greater than 2V(~/0, 36 Q/o). 


8-6 Geometric interpretation of the results of Chapter 8 


The inequalities (8-2) and (8-3) show that the smaller the distance between the points 
of the signal curve corresponding to parameter values differing by the amount 2€, the 
smaller A, and the larger the probability that the error exceeds the value € . Thus, the 
smaller this distance, the smaller the noise immunity, This situation is quite natural, 
since the smaller the distance between the points corresponding to the two signals, the 
larger the probability that the signals will be confused for each other and will be incorrectly 
reproduced by the receiver, as a consequence of the addition of noise and the resulting 
displacement of the points. For the cases of pulse time modulation and frequency modula- 
tion, the value of ais and of this distance at first increase in proportion to € , and then 
stop growing and even begin to decrease from IL, « = 4.5 0n (see Fig. 8-1), This 
property of the modulation allows us to increase the length of the signal curve and thereby 
increase the noise immunity without increasing the signal energy, but it is also responsible 
for the appearance of anomalous errors. In geometric terms, the problem of increasing the 
noise immunity in the presence of weak noise, without increasing the energy, length or band- 
width of the signal, reduces to increasing the length of the signal curve without having the 


curve leave a certain hypersphere (the radius of which is determined by the maximum 
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energy given to the signal), and without increasing the number of dimensions of the space in 
question, It is clear that a signal curve of any length can be constructed which lies within 
the volume of any hypersphere, However, when the length of the curve is increased, the 
distance between separate "twists" or sections of the curve must decrease, which perforce 
increases the probability of anomalous errors. Thus, the law which we noted in a special 
case is obviously valid in general, namely an indefinite increase of the noise immunity in 
the presence of weak noise without increasing the specific energy, duration, or bandwith 
of the signal is necessarily accompanied by an increase in the probability of anomalous 
errors, If we increase the duration or bandwidth of the signal, we thereby increase the 
number of dimensions of the space in which the signal curve lies. In this case, we can 
increase the length of the curve without leaving a given hypersphere and without bringing 


different sections of the curve close together, 
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PART IV 


TRANSMISSION OF WAVEFORMS 


CHAPTER 9 


GENERAL THEORY OF THE INFLUENCE OF WEAK NOISE ON THE TRANSMISSION 
OF WAVEFORMS 


9-1 General considerations 


In communication engineering one deals in many cases not with the transmission of 
messages which can take on discrete values (as, for example, in the case of telegraphy), 
nor with the transmission of separate parameter values (as, for example, in the case of the 
transmission of separate measurements by telemetering), but rather with the transmission 
of time functions, which can vary continuously and can take on an infinite number of forms 
(as, for example, in the case of telephony). We shall consider this last type of transmission 
in Part IV, 

To simplify our considerations, we shall assume that the modulating waveform (a 
sound wave, say) is periodic with period T (this can always be achieved artificially by 
taking T large enough), and that the frequency spectrum of the waveform contains in effect 
only components indexed from i, to i,- In this case, we can write the modulating wave- 
form as 

ty £, 
F(t) = 2. (Ages Vz sin on it + Az, V2 cos as it) = x Ag Jy (t) (9-1) 
ad dy L =2) 


where the Ae are certain constants determined by the waveform, the Tp(t) are the ortho- 
normal functions defined by Eqs, (2-14), and 4, = 2i) - il, £, = 2i,. For simplicity, 
we shall assume henceforth that the function F(t) takes values lying between -1l and +1, 
and does not take any values outside this range. 

The waveform (9-1) is transmitted by using another waveform, which we shall call the 
signal (i.e. , the modulated waveform), Since we have assumed that the modulating wave- 
form is periodic, we can also assume that the signal is periodic. Inasmuch as the modulat- 
ing waveform (9-1) is completely determined by L, - 4, + 1 parameters, the signal 
must depend on these parameters ry: Thus, in general, the signal can be represented by 


the expression 


(Agro Ag st) (9-2) 
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For brevity, we write this expression on occasion as 
Ayit) (9-3) 


The noise Wa yf) is added to the signal, so that the received waveform has the form 
3 


X(t) = A p(t) + W 


Ya) (9-4) 


We assume, as before, that the function Wa,v is periodic, and moreover, that it has the 
same period T as the period of A p(t). Clearly, we can always assume this, since in both 
cases the same requirement is imposed on the period, namely that it be sufficiently large. 
When the waveform X(t) is received, the receiver has to reproduce F(t) with as great 


accuracy as possible, 


9-2 The influence of weak noise on the modulating waveforms 


As we specified, the modulating waveform is completely determined by the parameters 


Ap Soares Ay . Obviously, in reproducing F(t) the receiver thereby reproduces the given 
1 2 
parameters, We represent the waveform (9-4) as 


n 
X(t) = es x, C(t) (9-5) 


where the C, (t) are some system of orthonormal functions, Clearly, the parameters 
reproduced by the receiver are functions of the quantities xX, characterizing the received 


waveform X(t). Thus we can write 


Aye Byes) Ae hsh, ee 


The form of these functions depends on the modulation systems and the receiver. If the 


received waveform receives an increment 
n 
dX(t) = y Cy (t) dx (9-7) 
k=l 


then, obviously, the parameters Ap receive increments 


3 
n 
dry = ey 3 dx, = ty (t) dx(t) (9-8) 
where 
n re) 
= Cc = 
Lott) oe = At) (9-9) 
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We assume that the receiver correctly reproduces the modulating waveform in the 
absence of noise. Let the transmitted waveform be changed in such a way that Ag receives 
an increment drg. Then the signal and therefore the waveform X(t) (since noise is 


absent) must receive the increment 
dX(t) = Dglt) dAg 


where dA L(t) 


Bh tee hs 9-10) 
Dg (t) Ny 


It follows from (9-8) and the fact that the receiver must reproduce the modulating waveform 


without error that the relation 


L 2 
1s valid, 1. Ces that 


Tg D = 1 (9-11) 


On the other hand, the remaining parameters a; (i #20) are not changed, so that 


da, = L(t) dxX(t) = Ltt t) Dg (t) dry = 0 


whence 


L(t) Dg(t) = 0, i #42 (9-12) 


Thus, for any receiver which correctly reproduces the modulating waveform in the absence 
of noise, Eqs. (9-11) and (9-12) must be satisfied, where & is any integer from £, to 
L. 

Now suppose a modulating waveform is transmitted which is characterized by the 
parameters Ag , and let the noise Wav) (with sufficiently low intensity) be added to the 
signal which is used to transmit the waveform. Then, due to the action of noise, the 
received waveform receives an increment 


dX(t) = UE yf) 


? 


as a result of which the parameters of the waveform reproduced by the receiver receive 


increments 
2 
dry = Lgl WT - GP) V Ly) @ (9-13) 


and are equal to 2 + dr, . Thus, the increments dAp resulting from the action of noise 
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are random variables which obey a Gaussian law. The smaller 


2 
V La tt) (9-14) 


the smaller the increments, and therefore the smaller the errors given by the receiver. If 


we choose the La lt) in such a way that they satisfy Eqs. (9-11) and (9-12), and such that at 
the same time the values (9-14) have the least possible values, then the receiver character- 
ized by such Lg(t) will give the least reproduction error for sufficiently weak noise, In 
the next section, we shall find the optimum values of L(t); later on, we shall show that 


the receiver having these values of Lylt) exists, at least in principle, 


9-3 Conditions for the ideal receiver 


We now find the conditions which the L(t) must satisfy, i.e., the conditions which 
the receiver must satisfy, in order that weak noise should produce as small errors as 
possible in the transmitted waveform. We shall call the receiver which satisfies these con- 


ditions ideal. We shall consider the case where all the 


dAp(t) 
Dp (t) = (9- 15a) 
L 
3 Ag 
are mutually orthogonal, and where 
2 2 
D5,_,(t) = Dj(t) (9-15b) 


This case is the most interesting, since, as we shall see, these conditions are satisfied for 
all the modulation systems used in practice. The presence of these conditions will greatly 
simplify the subsequent considerations and the final results. 


Any function, including Lglt), can be written as 


D, (t) 


Ly (t) = ae By (t) (9-16) 


D*(t) 


Lg 


where By (t) is an as yet unspecified function. Substituting this quantity in Eq. (9-11), 


we obtain 


D 

y(t) 

—_——. + Dy lt) Bt) = 1 
= L 
Dp (t) 


whence 


Substituting Eq. (9-16) into Eg. (9-12), we obtain 


Dp tt) D.(t) 
ct Dg (t) B.(t) = 0 


2 
Dele) 
However, since i # LZ, so that D,{t) Dy) = 0, we must have 


Dy tt) B.(t) = 0 


Thus, in order for Lg(t) to satisfy Eqs. (9-11) and (9-12), it is necessary and sufficient 
that each B.(t), i= L,; itexs L,, be orthogonal to all the Dy (t), for b-h,, ei L,. 
According to (9-16) we have 


2 
D,(t) = : 
Lp(t) = ae ee Bp(t) | = ppt Sec By (t) (9-17) 
2 
Dpi(t) Dg it) 
since, as we have explained, Dg (t) and Bg (t) must be orthogonal, This expression is 
obviously a minimum for By (t) = 0, whence it follows that for the ideal receiver 
Dy (t) 
Dy) = (9-18) 
Z 
D, (t 
| ) 


where Dg(t) is defined by Eq. (9-10). 


9-4 Means of realizing the ideal receiver 


We now show that the receiver which, when the waveform X(t) is received, reproduces 


the value of the function which minimizes the expression 
2 
R = [X(t) - Az(t)] (9-19) 


is ideal in the sense formulated in the preceding section. In fact, when a modulating wave- 


form F(t) is transmitted in the absence of noise, we obviously have 
x() = Ap Ct) 


and Eq. (9-19) has its least possible value of zero for the case where Az(t) and Ap (t) 
fo) 
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coincide, and the waveform F(t) reproduced by the receiver is F(t). Thus, the receiver 
in question does not introduce errors in the absence of noise. F(t), and therefore R, isa 
function of the parameters Ag . We stipulated that the waveform F(t) reproduced by the 
receiver is to give the minimum value of the expression R., Therefore, the partial 


derivatives of R with respect to Ag must vanish. We obtain the condition 


oR DME Ae EE 
Ape oe ee ee re eae! 
where 
dA_(t) 
7 F 
clea 


If the received waveform receives a small increment AX(t), then, obviously, Ap(t), F(t) 
and Ay must also receive increments if the expression for R is again to be a minimum, 
Suppose the parameters Ay receive the increments Ady: then Ay(t) receives the 


instrument 


£, 
AAW(t) = J Dyl(t) AA (9-21) 


which means that we have 
aa aaa (0 ea 
2 


R= [X(t)+ X(t) - A(t) - Pe Dg (t) bag 1” (9-22) 


The values of the increments AAg must be such that the expression R again has a minimum 
value, Therefore the partial derivatives of R with respect to Ady must vanish, so that 
SSS SS Sg 

2 


dR 
-2 [X(t) AX(t) - AL(t) - D (thAA, }Dp(t) = 0 
(t) + _ ms, 9, 2 


3A Ay = 


Moreover, taking into account Eq. (9-20) and the fact that the Dg(t) with differenti indices 


are orthogonal, we obtain 
AX(t) Dg(t) - Dg tt) dAg = 0 


whence 


QB X(t) Dy (t) 
aay = ————- (9-23) 
Dg (t) 
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The smaller AX(t) and Arp» the more exact Eq. (9-21) is. Letting these quantities go 
to zero, we arrive at the condition characterized by Eqs. (9-8) and (9-18), Therefore, the 
receiver which reproduces the waveform F(t) minimizing Eq, (9-19) has no error in the 
absence of noise, and gives the minimum possible error in the presence of weak noise. 


Thus, this receiver is ideal in the sense established in Section 9-3. 


9-5 The error for ideal reception 


We now determine the amount of error given by the ideal receiver when weak 


fluctuation noise is added to the signal, Suppose a waveform 


£, 
F(t) = ge og Ip (t) (9-24) 
cia | 


was transmitted, Then, in the absence of noise, the received waveform is X(t) = Ap (t), 


and the ideal receiver reproduces the waveform F oft) determined by the parameters A of" 
When the weak noise W yt) is added to the vee: the received waveform is changed by 
an amount dX(t) = We nO, and, according to Eqs. (9-8) and (9-18), the parameters Ag 5 


which characterize the waveform reproduced by the ideal receiver, receive increments 


D, (t) dX(t) Dz(t) W(t) o8 
dy pS SC Sree Rs ee (9-25) 


a ar 2 ———— 
Dy (t) Dg (t) V2T Dy (t) 


It should be noted that the random variables 89 with different indices are independent, 


since the Dp (t) with different indices are orthogonal. Thus, the waveform reproduced by 


the ideal receiver has the form 


4 4 
2 2 6% Iyit) 
LL 
mp (Aog + 4Ag) lt) = Folt) o Se 


F(t) 


2T D“(t) 
= F(t) + W(t) (9-26) 
where 
i 

a = Oo ey, Cee an 

*(t) = —_ — i 

(t) Bs, : (0,;_) Sin =p it + 04, cos Se it) 

ay Do, (t) 
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Comparing this expression with Eq, (D-3), we see that due to the action of the noise which is 
added to the signal the receiver adds to the modulating waveform F(t) normal fluctuation 


noise with an intensity at frequency i/T equal to 


Ad 
o* (=) = [om = (om (9-27) 
2 2 
Ve D2; y(t) D2; (t) 
where 
0A _(t) 
F 
Dy (t) = 
L ary 


We shall henceforth call this normal fluctuation noise the noise at the receiver output. This 
intensity of the noise at the receiver output is the minimum possible and characterizes the 


optimum noise immunity for a given modulation system, In the case where o- *(i/T) does 


not depend on i, we omit this index and write o-*, 


9-6 Brief summary of Chapter 9 


We call ideal the receiver which exactly reproduces the modulating waveform in the 
absence of noise, and gives the best approximation to the modulating waveform in the 
presence of weak noe: The ideal receiver reproduces the waveform F(t) which minimizes 
the quantity R given by (9-19). When reception is with the ideal receiver and the noise is 
weak, the reproduced waveform differs from the modulating waveform by the fluctuation 
noise with intensity given byEq. (9-27). In drawing these conclusions, it was assumed that 


the functions Dg(t) = 3A 5. (t) [dX are orthogonal for any pair of different indices, and 


2 2 
that D5; _ y(t) = Do, (t). 
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CHAPTER 10 


DIRECT MODULATION SYSTEMS 


10-1 Definition 


By direct modulation systems we shall understand systems in which the modulating 
waveform (message) F(t) enters directly as a parameter into the expression for the trans- 


mitted signal. In this case, we can write the general form of the signal as 

Apt) = A (F(t), t] (10-1) 
Examples of direct modulation systems are amplitude modulation, where the signal can be 
written as 


A, (t) = US {14+ MF(t) ] cos(u) t + 0.) 


phase modulation, where the signal can be written as 


Ap(t) = Us cos [wt + mF(t) + 051 
etc. Frequency modulation, where the transmitted signal is written as 


AL(t) = U,coslat SL fri) dt] 


does not belong to the direct systems in the sense of the terminology of this book. Since the 
modulating waveform F(t) appears behind the integral, we shall call this kind of modulation 
integral modulation. Single sideband transmission is also not a direct system, since in this 
case also the signal cannot be expressed analytically in terms of the modulating waveform 
F(t), In Chapter 11 we shall study pulse modulation systems, which are also not classified 


as direct systems. 


10-2 Derivation of basic formulas 


Since by hypothesis the modulating waveform F(t) can be expressed by Eq. (9-1), for 


f 
os 


a direct modulation system we can write the signal as 


Api(t) = A[F(t),t] = A 


rp p(t), t 
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whence 
BAn(t) BARE) 
Dp {t) = aaa = mer ae Ipit) 


We also assume that the function [oA p(t)/ ary? contains only sinusoidal components with 
frequencies greater than Lie, i.e. , greater than twice the maximum frequency of the 
sinusoidal components of the modulating waveform F(t); this condition is usually satisfied. 


Then, according to Eq. (2-26), we obtain 


Dp) = (OARW/SFIS I(t) = (BAR (t)/ aF I* 


(10-2) 


It follows from these equations that the conditions (9-15) which were imposed on the Dy (t) 


ft 


[3A (t)/ 9F I Iy(t) L(t) = 0 


are satisfied in this case, and we can use Eqs. (9-26) and (9-27). It is a consequence of 
these equations that, for the kind of modulation system in question, we have at the output of 
the ideal receiver not only the modulating waveform F(t), but also normal fluctuation noise 
added to it. This noise has a uniform spectrum and an intensity which, according to Eq. 


(10-2), is equal to 


a om 


i aay (10-3) 
“V (aapw/ar FP 


It is all right to assume that the noise has the same frequencies as those contained in the 


waveform F(t), since any other frequencies can be filtered out of the receiver output, 
10-3 Optimum noise immunity for amplitude and linear modulation 
In amplitude modulation the signal can be represented by the expression 
Ayl(t) = US [1+ MF(t) ] cos(Wt + i) (10-4) 


where M is the coefficient of modulation, since we agreed to assume that -1 <F(t)< +1. 


It follows from this formula that 


—F— = UUM cos(wt + 9) (10-5) 


This is a high-frequency waveform with frequency wW fan ,» so that the restriction imposed 


in Section 10-2 is satisfied. Moreover 
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2 l 
(dA, (t)/ oF J a te es 
so that, as a result of noise, at the output of the ideal receiver we have normal fluctuation 
noise with intensity 
10-6 
o* = SZ O/UM (10-6) 
Here, and in what follows, it is assumed that the gain of the receiver is adjusted so that 
the waveform produced at its output by the signalis F(t). For convenience in comparing 
this with other methods of modulation, we express U. in Eq. (10-6) in terms of the effec- 
tive value of the signal voltage taken for the cases F(t) = 0 and F(t) = cos-tLt. For 
F(t) = 0, we obtain 


uz = a%(o,t) =2 v2 
eo 2 0 
and for F(t) = costLt, we obtain 
2 =) we ey eee 1 2 
Uim 7 & (cos flt,t) =5 UG (+ 5M") 
whence 
2 
; ne a+ o* 
ee ee ee (10-7) 
MU M"U 
eo em 
The maximum noise immunity is obtained for M = 1. In this case 
z 2 2 
aoe a oo (10-8) 
U 2U 
eo em 
For linear modulation, the signal can be written as 
A(t) = UL cos(W.t + 01) + USMF(t) cos(a) t + 9.) (10-9) 


The amplitude modulation analyzed above, the so called quadrature modulation, and also 
suppressed carrier transmission using both sidebands are special cases of linear modulation. 
It is easy to see that Eq. (10-5) is also valid for this kind of modulation, which means that 

in this case at the output of the ideal receiver, in addition to the modulating waveform F(t), 


there is normal fluctuation noise, with intensity given by Eq. (10-6). 
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10-4 Optimum noise immunity for phase modulation 
= eee or phase modulation 


For phase modulation, the signal can be written as 
A(t) = US cos (wt + mF(t)] (10-10) 


where m is the modulation index, For sucha signal, we obtain 


3A p(t) 
ae Ujm sinfwt + mF(t)] 
whence 
3A A(t) 7 usm? uem* 
se = 5 - 5 cos [ Zw t + 2mF(t) ] 


In the case where w is large enough, this waveform has no low-frequency components, 


so that the condition (10-2) is satisfied. Moreover, we have 


2.2 
eo Ujm 
[dA ,(t)/ oF ] nat iat 


which means that, due to the noise, at the output of the ideal receiver we have, in addition 


to the modulating waveform F(t), normal fluctuation noise with intensity 


ox = f2 o/U jm (10-11) 


For phase modulation, the effective value of the signal equals 


ue = A%(0,t) = A(costLt,t) = su 
Introducing these values into Eq, (10-11), we obtain 


o* = o/mU, (10=12} 


We see that for phase modulation, the optimum noise immunity is as many times greater 
than the optimum noise immunity for amplitude modulation as m is greater than M, Since 
for amplitude modulation M cannot be larger than unity, whereas for phase modulation m 
can be much greater than unity, we can obtain greater optimum noise immunity for phase 


modulation than for amplitude modulation. 


10-5 Noise immunity for amplitude modulation and ordinary reception 


We now compare the optimum noise immunity for amplitude modulation (obtained in 


Section 10-3) with the noise immunity obtained for this kind of transmission when using an 
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ordinary receiver. If before the detector there is a filter which passes the signal frequen- 
cies, then at the filter output the noise voltage has the form given by Eq. (B-6), so that 


the sum voltage acting on the detector equals 
Us [1 +MF(t)] cos wt +2 wt att) cos wt t+ Vz Wi nt?) sin wot 


(10-13) 


The amplitude of this waveform equals 
UL i= ~/ gu. (1 + MF(t) ) +vV¥2 WwW! (7 q Wie (t) (10-14) 
m ° l,n l,n 


. ; : 2 
If the noise is sufficiently small compared with the signal, then quantity wy n can be 
) 


neglected compared with the square of the term in curly brackets. Then we obtain 


Ui ee U. [1+ MF(t)] + V2 Wi att) (10-15) 


If we assume that a linear detector is used at the receiver, then the a.c. component at the 


detector output is equal to 


K [U, MF(t) + V2 wr y(t) (10-16) 


If at the receiver we use a synchronous detector, which responds only to the cosine compon- 
ent of the voltage (10-13) applied to the detector, then this result is exact even for large noise 
intensity. If the gain of the receiver is chosen so that the waveform at its output is equal to 
F(t) in the absence of noise, then in the presence of noise, according to Eq. (10-16), the 
waveform is 
Vz us 

F(t) “MU, ae (10-17) 
Since, as shown in Appendix B, wr it) is normal fluctuation noise with intensity OF then at 

) 
the receiver output there is added to the transmitted waveform the normal fluctuation noise 


with intensity 
o*= V2 o7/MU, (10-18) 


Thus, we see by comparing Eqs, (10-6) and (10-18) that the influence of noise is the same 
for the real receiver and for the ideal receiver. We can conclude from this that for 


amplitude modulation, the ordinary receiver with a linear detector provides the optimum 


noise immunity in the presence of weak noise. Hence, in the case of signals of the form 


(10-4) and for weak noise, no improvements can give a noise immunity higher than that 


given by the ordinary receiver with a linear detector, 
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The same result is also obtained when we investigate other linear modulation systems, 
e.g. , quadrature modulation and two-sideband, suppressed carrier transmission. In these 
cases it also turns out that the reception normally used with these methods provides the 


optimum noise immunity. 


10-6 Noise immunity for phase modulation and ordinary reception 


For phase modulation with the signal given by Eq. (10-10), taking into account added 


noise, we obtain in the receiver (after the r.f. or i.f. filter) the waveform 
" ' : . 
U, cos (wot + mF(t) ] +2 we cos at + V2 wi le) sin wot (10-19) 


as follows from Appendix B. To simplify the calculation, we consider only the case where 
the modulating waveform is small and mF(t) << 1, In this case, for weak noise, when we 
can take wy nit) << U. and wi nit) < US» we can represent the sum of the waveforms 
? ? 
(10-19) by one waveform 
U_, cos Cat + P(t) ] 
where 
7 V2 ws 
P(t) = mF(t) - y- Wy 

If this waveform is applied to a phase detector which reacts only to its phase and not to its 


amplitude, then after the phase detector we obtain the waveform 


F(t) + a wy nit) 
oO > 


after choosing the gain in the required way. Since, as shown in Appendix B, wi nit) is 
? 


normal fluctuation noise with intensity o7, then in this case, after the phase detector thereis 


added to the transmitted waveform F(t) the normal fluctuation noise with intensity 
2 o/mU , ( 10-20) 
Thus, we see bycomparing Eqs. (10-11) and (10-20) that the method of reception examined 


here provides the optimum noise immunity in the presence of weak noise, at least for 


small modulation indices. 
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10-7 Noise immunity for single-sideband transmission 


In this section we study the noise immunity for single-sideband transmission. This 
transmission system does not belong to the direct systems, but is discussed here for con- 


venience. We now find the influence of noise for this kind of transmission and reception with 


the ideal receiver. If the modulating waveform 
i 
F(t) = 7 ( V2 sin it + A,, WE cos it) 
se: “4. 221-1 danas 2i aa 
“ol 


is transmitted, then if the upper sideband is used, the signal has the appearance 
u _ on, an. 
A, (t) = US 2 [Pei-r V2 sin(=_— i+ w,)t + Aa V2 cos(—p- i+ w,) (10-21) 


where Ww fen is the carrier frequency. From this we obtain 


= . 2k 
Do; ,(t) = US 42 sin(—> i +W,)t 


is an. 
Da; (t) = UL 2 cos(—ar i + ),)t 
Therefore 
Z 2 2 
Pee eso ee Ue 
and 


D.() Dstt) = 0, iv j 


Thus, the general formula (9-27) is applicable to the system in question, so that the noise 


intensity at the output of the ideal receiver equals 
ae tes / fey 2 
o* -o/ V ppt) = o/v? (10-22) 


We obtain the same noise immunity for reception on the receiver usually used to receive 


single-sideband transmission, 
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CHAPTER 11 


PULSE MODULATION SYSTEMS 


11-1 Definition 


By pulse modulation systems we shall understand systems in which, instead of con- 
tinuously transmitting a modulating waveform F(t) using the signal Ap(t), we transmit only 


the separate instantaneous waveform values 


eee 9 F(-2%), F(-%), F(0), F(®), F(2@), oe (11-1) 


taken at instants of time separated from one another by the amount %. We achieve the 
transmission of these instantaneous values by using separate signals (pulses) which follow 
one another in sequence. In doing this, we can use any of the methods of transmitting 
separate parameter values discussed in Part III]. In this case, the transmitted (i.e. , modu- 
lating) quantities are the instantaneous values (11-1). 

To transmit the instantaneous value F(0), we use a signal AU, »¢) beginning at 
t = 0, where we have set oy = F(0), To transmit the instantaneous value F(k%), we use 


a similar signal AU, 5 t - Uk) beginning at t = Yk, where 


L = F(k@) (11-2) 
Thus, we achieve the transmission of the waveform F(t) by using a signal 


(T/2%)-1 


A,(t) = AU, t - Gk) (ie 3) 


k =-T/2% 
The choice of the limits in this sum results from the fact that all processes studied in this 
book have to lie in the interval (-T/2, +T/2). Weassume that T is chosen in such a way 


as to make T/2VU an integer. 


11-2 A way of realizing the pulse modulation system 


We now examine a possible way of realizing the pulse communication system, and ex- 
plain the basic relation which makes the system realizable. The means of obtaining the 
signal pulses in the transmitter is in principle very simple, and can be schematically 
achieved as follows: At instants of time which are multiples of UY, a switch closes a 
circuit on which there acts an e.m.f, proportional to F(t). Then current pulses with values 
proportional to the instantaneous values (11-1) flow in this circuit. These current pulses 
act on a modulator, and change the form of the r.f, pulses sent to the receiver by any of 
the methods studied in Part II], When the r.f, pulses arrive at the receiver, the instantan- 
eous values (11-1) sent by the transmitter are first restored, and then short pulses 
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proportional to these instantaneous values are produced. These short pulses can be written 
as 


F(kv) P(t - ke) 


The voltage produced by all these pulses is 


(T/2%)-1 
F(kt) O(t - kt) (11-4) 
k == [2% 


Here we do not take into consideration a possible constant delay of the pulses at the receiver 
with respect to the pulses at the transmitter, 
We assume that F(t) is a continuous function and that Q(t) =0 for t < 0 and 


t>+e. Then the equation 


F(kt) Gt - kv) = F(t) ®t - kB) 


is valid with arbitrarily great accuracy, if ¢ is sufficiently small, In fact, @(t - kv) is 
different from zero only for values of t which lie in an arbitrarily small interval 

(kt, k&% + €) in which we can assume that F(t) = F(k%). Taking account of this fact, we 
can write Eq. (11-4) as 


(T/2T)-1 é (T/2%)-1 ' 
F(kt) O(t - kV) = F(t) (t- kt 
k ee ee 


(11-5) 


= F(t) do + F(t) d, cos(q).t + 01) + F(t) d, cos(2w t + 0,) tees 


where WJ, = 2n/7, and ds dj, d,, ... arecertain constants. The last expression is 
obtained by expanding the sum 
(T /22)-1 
Bit - kV) (11-6) 
k =-T/2¥ 


which is a periodic function of ©, as a Fourier series. Let the highest frequency entering 
the waveform F(t) be fnax’ Obviously, the highest frequency entering into the first term 
of the series is equal to this quantity. The second term of the series is an amplitude 
modulated waveform and can be decomposed into sinusoidal components consisting of the 
carrier and sidebands, where the lowest frequency of a component is obviously (w/2n) 


~ fax 
Suppose that the highest frequency Bo of the first term is less than the lowest frequency 


In the third term, the lowest frequency is obviously (2 w/e x) - tes 255 and so forth. 


of the remaining terms, i.e, 


if = < (w/2 x) - i 


ma (11-7) 


ax 
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or 


1 Wo 
Zo) Be pe eet ens (11-8) 
or 
t< on) (11-9) 
max 


Then, because of the frequency separation, it is clear that the first term of (11-5) can be 
completely filtered out from the other components, so that we can obtain F(t), Thus, using 
the method described, i.e, , the pulse representation (11-4) with a filter or harmonic 
analysis, we can reproduce the waveform F(t), if only the frequency of the pulses is greater 
than twice the maximum frequency entering into the waveform F(t), or, what amounts to the 
same thing, if the distance Y% between the pulses is less than half the smallest period of 


a sinusoidal component of F(t). 


11-3, Optimum noise immunity for the pulse modulation system 
We now determine the optimum noise immunity of the pulse modulation system, start- 
ing from Eq. (11-3) and the general formula (9-27), We obtain 
dAp(t) (T/2T)-1 dAla,, t- Tk) AF (Ke) 


—.—_. (11-10) 


D (t) => —_ TS 
4 8Ag kk atT/2t 8 x aAyQ 


We assume that pulses AV, t - k&) with different k do not overlap, so that at any 
instant of time t only one of the terms of the sum (11-10) can differ from zero. In this 


case, the separate terms of this sum are orthogonal, and we obtain 


SS (T/2T)-1 SS 
(kv) _ OF (kv) 
D_(t) Dg (t) = dA(A,, t - ke)/d a 11-11 
oe ie ealepine ae a oe dAm dry : 
Moreover, clearly 
———_——_——; 


[BAY T= KRM I = LAY WMT = Aye (1-12) 


We shall assume for simplicity that this quantity does not depend on the value of je 3 
this is the case, e.g., in all the examples analyzed in Part Ill. In the cases where this 
quantity depends on the value of fe» the intensity of the noise process at the receiver 
output will depend on the modulating waveform F(t), and the calculation of this intensity 
becomes complicated. However, if in this case we look for the noise intensity at the 
receiver output for F(t) = 0, then the results of the calculations are valid, we need only 


replace the expression 
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AT Yoo) by its value for Ke = 0. Taking account of (11-12), we obtain 


(T/2%)-1 OF(kv) AF(k®) (11-13) 


12 PI ZES Mattia Ae EES 
Pate ne pee « a£T/27% 2AM dry 


Moreover, from Eq. (9-1) we obtain 


SE(Ke) = V2 cos 2m. ik © (11-14) 
3A; ey 

_OF(KT) sin on ik® 

dA2i-1 


Substituting these expressions into the sum (11-13), we obtain 


D_ (t) D,{t) = 0 for m #4 
m g 
(11-15) 
ae: 12 
Dg it) % 7 Ay hot) 


Thus, according to Eq. (9-27), at the output of the ideal receiver, in addition to the repro- 


duced waveform F(t), we obtain fluctuation noise added to it, with a spectrum equal to 


es et (11-16) 
oy are sar 
T Ayes t) 


We know from Section 6-6 that when a parameter w is transmitted using the signal A(g,t), 


we obtain the least possible mean square error § oF given by Eq. (6-40), when reception 


m 
is with an ideal receiver. It follows from the form of these equations that the noise intensity 


at the receiver output can be expressed as 
ox = 2087 (11-17) 
mm 


We see that the normal fluctuation noise which the added noise produces at the output of the 
ideal receiver has uniform intensity, just as in the case of the direct modulation methods. 
The intensity of this noise becomes larger when the minimum mean square error @ ek 
for transmission of the instantaneous values F(k%@) of the transmitted (modulating) wave- 
form becomes larger. Thus, the problem of raising the optimum noise immunity for the 
pulse modulation system reduces to decreasing the minimum mean square error obtained in 
transmitting the instantaneous values, All that was said about this in Part II] is applicable 
in the present case. By decreasing “© , i.e., by decreasing the number of signal 


pulses, we can decrease O6*, but the average signal power is thereby increased, 
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In what follows, we shall need to know the effective value U. of the signal when 
° 


F(t) = 0, which for the pulse modulation system is given by 


5 Cel 2p - Ss eee (T/2%)-1  ——————-— 
Us Actt) = | ry A(0,t - Ko | 7 2 A“(0,t - kv) 
= TT/2% k =-T/2U 
1 ERD ate 
5 AO i 
= (0, t) (11-18) 


according to Eq, (11-3). Here we used the fact that the waveforms of the separate pulses 


do not overlap, and are therefore orthogonal as this implies. 


11-4 Noise immunity of the receiver analyzed in section 11-2 


In this section we investigate the noise immunity of the pulse modulation receiver 
which uses the principle of operation studied in Section 11-2, and we compare this noise 
immunity with the optimum noise immunity. In doing this, we assume that the first part of 
the receiver in question, which reproduces the instantaneous values from the received 
signal, operates ideally. In Section 6-5, it was shown that when weak noise is added to the 
signal, the transmitted quantities are reproduced by the ideal receiver with errors which in 


the given case, according to Eqs. (6-29), (6-36) and (6-38), are for the k-th pulse equal to 


$= Tw ey Wot = §  % (11-19) 


where 


= ! 12 
L(t) = Ay(pst = ka) ALs pot ~ ke) 


6, is anormal random variable, and 


k 


o 


Sam ee a ay, 
Say eis. oe 
2T Ay ot - k®) 


is the mean square error with the ideal receiver, Since the pulses which are used to trans- 
mit the various instantaneous values are by hypothesis non-overlapping, the L(t) with 
different indices are mutually orthogonal. Therefore, according to (2-60) and (2-61), the 


O. are mutually independent. Moreover, since we assumed in Section 11-3 that 


Z , “8 2 
An (pe, t - k@) does not depend on Ww, we find that the quantities L(t) and therefore also 
35 mm 7 net depend on w . 


Due to the action of noise, the receiver reproduces the values F(k%) + 3 am 9. 
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instead of the instantaneous values F(k%). According to Section 11-2, in order to use these 


values to restore the waveform F(t), we form a system of short pulses, which in this case 


has the form 


(T/2TZ)-1 1B es 
F(k@) + Q t-k& 
k Las mse 
(T/2%)-1 7 F (T/2)-1 aa 
= F(kt) (t - k&) + 8 t-kt™ 11-20 
k ee mim ee poe S 


If in this expression we leave only oscillations with frequencies less than 1/24, then, as 
shown in Section 11-2, the first term of this expression equals the quantity do F(t), where 
qd, is some constant, We now show that under these conditions, the second term is normal 


fluctuation noise with intensity equal to 


V27T Sm dy (25-21) 


for the frequencies from 0 to 1/2%. We first find the cosine component of the second term 


at frequency n/T; it equals 


+T/2 (T/2%)-1 ‘ 
C = 2 5 8. Q(t - kt) cos nt dt 
oS Te SR Oe 
-T/2 a 
2 
26 (T/2t)-1 +T/ 
4 eo 8 5 Q(t - k%) cos cae nt dt 
LE ote capa. & T 

~ -T/2 


Since Git - k¥) is different from zero only in the immediate neighborhood of t = kT, 


we have 
+T/2 
2n a 2x 
J Gilt - kT) cos —7 nt dt = a cos FO Uke 
-T/2 
where 
+T/2 +T/2 
a= J Q(t - kt) dt = J Q(t) dt (11-22) 
-T/2 -T/2 
so that 
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2 Sam (T/2T)-1 3 
Cle — a 8, cos SF nk& 
== T (26 


According to Eq. (2-74), taking into consideration the fact that the o. are independent 


normal random variables, we obtain 


C= a aNT/2% o_. (11-23) 


where pee is a normal random variable, inasmuch as 


(T/2v)-1 (T/2T)-1 
2 2n nk®@) = — 


1 1 4 
( us 


=> + = cos 
ea2T/2zee* = 


k =-T/2% 


since the sum of cosines is zero for n/T < 1/2U. In the same way, the amplitude of the 


sine component at frequency n/T is equal to 


2d 
mm 
Sy =— 4 VT/2T Doe (11-24) 
Using Section 2-5, it.is not hard to show that the random variables 8. 85. 8 2? 952° aos 


are mutually independent. 


We now find the quantity qd. which is the constant component of the series (11-6). 


It equals 


+T/2 
1 (T/2T7)-1 19° 
d acl 7 j l(t - kt) dt ~ 7p 5 ies (11-25) 
S _t/2. Ke =T/2t 


whence a = td... Bearing in mind all that has been said, and retaining in the second term 
of the waveform (11-20) only components with frequencies less than 1/2%V, we obtain the 
waveform 

(T/2%)-1 


Z 2n F 
1, (T/2%)-1) ~ Onin ee eG ae ae GOS sap eas See 


W am 


nt) 


(11-26) 
which, as follows from a comparison with Eq. (2-54), is the normal fluctuation noise with 
the constant intensity equal to (11-21), as was to be proved, 

If we aow choose the gain of the receiver in such a way as to make F(t) the waveform 
at its output in the absence of noise, then, clearly, the additional waveform which is added 
to the output waveform in the presence of noise is the normal fluctuation process with 


intensity 
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o * = V2 Son (11-27) 


m 


Comparing this result with that obtained in Section 11-3 for the optimum noise immunity, 


we arrive at the conclusion that the means of reception analyzed in Section 11-2 provides 
the optimum noise immunity, if in it we use the ideal receiver to reproduce the instantaneous 


transmitted values, In the case where a nonideal receiver is used for this purpose, the 


mean square error Sin in reproducing the instantaneous values is larger than 3 5 


mm 
and the intensity of the noise at the receiver output is increased by the same amount as 


compared with the ideal case. 


11-5 Optimum noise immunity for pulse amplitude modulation 


For pulse amplitude modulation the separate pulses are given by Eq. (7-1). Accord- 


ing to Eqs. (7-2) and (11-17), the noise intensity at the output of the ideal receiver is 


of Ss So (11-28) 
T B“(t) 


For convenience in comparing this with other kinds of modulation, we replace B”(t) in 


this formula by the effective value of the signal. According to Eqs. (7-1) and (11-18), we 


obtain 
2a EAPO st)s 2 PUBL 
eo z 7a 
Substituting this quantity in Eq. (11-28), we obtain 
o*= > (11-30) 
eo 


We now compare this value of the intensity of the noise process at the output of the 
ideal receiver with the same quantity for ordinary amplitude modulation, discussed in 
Section 10-3 and characterized by Eq. (10-8). As the comparison shows, the noise intensity 
at the output, and therefore the optimum noise immunity, of the two systems is the same. 
We saw in Section 10-5 that the optimum noise immunity for amplitude modulation in the 
presence of weak noise can be realized using the ordinary receiver. Therefore, the pulse 
amplitude modulation system cannot provide better noise protection for the same average 
signal power than ordinary amplitude modulation, regardless of the receiver, at least for 
weak noise and under the conditions for which the method of reception described in 


Section 10-5 is realizable. 


11-6 Optimum noise immunity for pulse time modulation 


Let the signal in this case be given by Eq. (7-9). According to Eqs. (7-11) and 


(11-17), the noise intensity at the output of the ideal receiver is equal to 


2 24 ee 


o* = —s (11-31) 
nT SLU; 


We now express this quantity in terms of ae using Eq. (11-18). In this case 


T AM yt) 7 sh ue 


whence 
2 
U 
2 2 2 pate 
ee < Cae 7 a ~ 2A se. (h=3e) 
Taking account of this value, we obtain 
2 12 & 
o* = —e ee (11-33) 
Hie. o 
o Ue 


for the case of pulse time modulation. It is clear that the noise immunity increases when we 
increase the time shift T, of the modulated pulses. Since this time cannot exceed ct, 
we have Oe: <T < 1/2f_, where fas is the maximum frequency of the transmitted wave- 
form. We give “Cx its maximum possible value by setting Tb, = 1/2f- In practice, 
TG, is always somewhat less than this value, so that for this value of Tb, we obtain a 
somewhat larger value of the noise immunity, which, according to Eq. (11-33) is determined 


by the quantity 


2-22 
48f° o 2 
Bee Sie SS a EE ee PAS Soe ts Hee of (11-34) 
2 172 zZ m 2 m 2 
tL Ue us a . e 


The quantity (2x f/f) shows how many times smaller the bandwidth 2f which the 
signal occupies for ordinary amplitude modulation is than the bandwidth (L/x it occupies 


for pulse time modulation. Comparing this formula with Eq. (10-8) which characterizes 


ordinary amplitude modulation, we see that the noise intensity at the output of the ideal 
receiver for pulse time modulation is approximately as many times less than that for 
amplitude modulation, as the bandwidth occupied by pulse time modulation is greater than 


that occupied by amplitude modulation. According to Eq. (2-57), for pulse time modulation 


with bandwidth 0 to fw? the effective value of the noise voltage at the output of the ideal 
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receiver is 


3/2 
anf 
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(11-35) 


11-7 Optimum noise immunity for pulse frequency modulation 
For this kind of modulation, the pulses are given by Eq. (7-37). According to Eq. 


(7-40), the noise intensity at the output of the ideal receiver is 


2 
es: aS (11-36) 
LL Uo % 


By Eqs. (7-39) and (11-18), the effective value of the signal waveform is in this case 


2 
ee ee em ed 
eo -em -e. 2c 

since 
2 1 2 
T Apt) = 5 UG, 


Substituting this quantity in Eq. (11-36), we obtain 


2: 12.67 


ox = — (11-37) 
ng Wie A 

Comparing this formula with Eq. (11-33), we see that they are completely identical. In 
both formulas LL/ m is approximately the bandwidth occupied by the signal, and TC, is 
approximately the time required to transmit one pulse. Therefore, all conclusions con- 
cerning this modulation system coincide with the conclusions concerning the pulse time 
modulation analyzed in the preceding section. In this case, we must also try to make TC, 
aslarge aquantity as possible. As before, the maximum possible value is e, HG. 4s 
Eq. (11-34) and the deductions from it are also valid in this case. Of course, the methods 
of combined modulation discussed in Section 7-9 are also available to raise the noise 


immunity in the presence of weak noise, 


124 


CHAPTER 12 


INTEGRAL MODULATION SYSTEMS 


12-1 Definition 


Systems such that the integral S F(t) dt,rather than the transmitted waveform F(t) 
itself, enters the analytic expression for the signal, will be called integral modulation 
systems. A well known example of such modulation is frequency modulation, where the fre- 


quency of the transmitted waveform can be written as 
W= &) + fLFit) 


where LL is the frequency deviation and F(t) is the transmitted waveform, the value of 
which by hypothesis varies within the range +1. As is well known, the analytic expression 


for the signal with this frequency is 


Ag(t) = Uj coslwt +f F(t) at] (12-1) 


It is apparent from this formula that this modulation differs from phase modulation, given by 
Eq. (10-10), in that it'contains the integral of the function instead of the function itself. Itis 
clear that very many different types of integral modulation can be produced. Todo so, it is 
enough to replace F(t) in any formula for the signal in direct modulation by the integral 

of F(t). 


12-2 Optimum noise immunity for integral modulation systems 


For the integral modulation system, the signal can be written as 


Apt) = A C frit) at, th= Aly,tl (12-2) 


where 


w= frat 


s 


ae te ae ye aera 
XD Ag V2 sin = it + A>, V2 cos <p it) dt 
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It follows from this that 
dA p(t) 7 dA, (t) sv 


aAy oy a4 


where 


= -s-~ V2 cos it 
dA25-1 me # 
oe T 
vee sar V2 sin “yr it 
Therefore, as in Section 10-2 
2 2 T é 2 
and 
D.(t) D = 0, k/ bh 


In proving these statements, it was assumed that the function [dAp(t)/ sy)? contains only 
sinusoidal components with frequencies higher than 2i,/T, i.e. , higher than twice the 
maximum frequency contained in the transmitted waveform F(t), Thus, the conditions (9-15) 
are valid for integral modulation systems, and we can use Eq, (9-27), Therefore, for these 


systems the noise intensity at the output of the ideal receiver is 


2 Bde 
o* y) - I i =< (12-4) 


V (34, (0)/3W)° 


so that the noise intensity at frequency f is 


o*(f) = aM Me ee as (12-5) 


2 
[oA (t)/ Jy) 


As we see from this formula, in integral modulation systems the noise intensity at the out- 


put of the ideal receiver increases in proportion to the frequency, as opposed to the modula- 
tion systems studied earlier. According to Eq. (D-9), the effective value of the noise pro-~ 


cess at the output is 
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Ve wy . vf oat = 2x f 
i BY eae. 
[a(t /3y ] S 


3/2 
> of 
- Bee ea Se (12-6) 
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12-3 Optimum noise immunity for frequency modulation 


We now apply the formula obtained in the preceding section to the case of frequency 


modulation, For frequency modulation, the signal can be represented by Eq, (12-1). Thus, 


applying the notation of the preceding section, we obtain 
Ap(t) = US cos(a) t +fLW) 
dAR(t)/ IW =- UM sin(wt + ML Y) 


As can be seen, the square of the last waveform does not contain any low frequencies if a), 


is sufficiently large. Moreover 


2 1 2 2 
[da p(y/ayy’ = 5 ue 0 


whence, according to Eq. (12-5), we have 


Y20 2nf 
* = - 
TAD Saya (12-7) 
For this kind of modulation, the effective value of the signal equals 
u* -_ ue = ue = i ue 
e To) em 2 fo) 
Therefore, in this case 
2nf Comm 
y= (12-8) 
o *(£) 7 Tw, 


According to Eq. (12-6), for this modulation the effective noise voltage at the output of the 


ideal receiver is 


pol ante! = 
2 2n m o m 
E w(t) = 22 ——— y— = 0,578 —3—— 4— (12-9) 
+vy3 LL ue te e 
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Comparing this kind of modulation with pulse time modulation and pulse frequency modulation 


with optimum noise immunity given by Eq. (11-34), we see that at the highest frequency tos 


the noise intensity at the output of the ideal receiver is approximately the same in both cases. 


In the case of frequency modulation, when the frequency is decreased, the noise intensity 


decreases, as opposed to the pulse systems, where it remains constant. This gives approx- 


imately twice as small a value of the effective noise voltage at the output of the ideal re- 
ceiver for frequency modulation as compared with pulse modulation, as follows by comparing 


Eqs. (11-35) and (12-9). A comparison of the noise immunity of the ideal receiver with the 
noise immunity of the real receiver which is usually used shows that the noise immunities 


are the same for frequency modulation in the presence of weak noise. 
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CHAPTER 13 


EVALUATION OF THE INFLUENCE OF STRONG NOISE ON THE 
TRANSMISSION OF WAVEFORMS 


13-1 General considerations 


In this chapter, we show how to evaluate the optimum noise immunity of systems which 
are used to transmit waveforms, in the presence of strong noise. A precise evaluation of 
the influence of noise in this case is often very difficult. The noise process at the receiver 
output may not even be normal fluctuation noise, and may depend on the transmitted wave- 
form. However, it is not hard to obtain an approximate evaluation of the influence of strong 
noise by using the maximum discrimination of the transmitted waveforms, which cannot 
be exceeded with any receiver, for the given means of transmission and the given noise 


intensity. 


13-2 Maximum discrimination of transmitted waveforms 


Let a waveform F  {) (a sound wave, say) be transmitted; in this case the trans- 
mitted signal is Ap (t). Let the noise Wav (t) be added to this signal, with the result that 
the receiver does nor reproduce the waveform F 1 {) at its output, but another waveform, 
which is a distortion of F(t) produced by the noise. If the waveform F(t) was trans- 
mitted instead of F(t), then the transmitted signal would be eh t). In this case, due 
to the action ofnoise, the waveform at the receiver would look like a perturbed version 
of the waveform F(t). The amount of distortion produced by the noise can be evaluated as 
the probability that by using the waveform reproduced by the receiver (a sound wave in this 
case) we correctly determine whether F(t) or F(t) was sent. This probability can be 
obtained experimentally, e.g., by using the following articulation experiment, which is 
suitable for the case of telephony: Sometimes let the sound waveform F(t) be sent, and 
other times let the waveform F(t) be sent, in an order which is unknown at the receiving 
end, but in such a way that on the average both waveforms are sent equally often, Suppose 
that at the receiving end a listener writes down each time which waveform, in his opinion, 
was sent. Obviously, in some cases he will write down the correct answer, and in other 
cases the incorrect answer, as can be ascertained subsequently. Then, for a sufficiently 
large number of trials, the number of correctly chosen sound waveforms divided by the 
total number of transmitted sound waveforms equals the desired probability. 

The maximum possible value of this probability for a given means of transmission, i.e. 
for given signals Ay (t) and aoe ae can easily be found theoretically. In fact, in 
Chapter 4 we found the probability that the ideal receiver correctly decides which of two 
signals known in advance was sent, when noise is added to the signal. Also we showed that 


no other means of reception can provide a larger value of this probability. If we correctly 
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determine which of two waveforms F(t) and F(t) was sent, by using the waveform at 
the receiver output, we thereby determine which of the signals A, (t) and A,(t) was sent. 
Therefore, the probability that we correctly decide which of the waveforms F(t) or F(t) 
was sent, by using the waveform at the receiver output, which is distorted by noise, cannot 
be greater than the probability that the ideal receiver correctly discriminates between the 


signals Ap (t) and Ap (t). According to Section 4-1, this latter probability is equal to 
1 2 


PoP = 1 - V(a) (13-1) 
where 


om 


as Avg [4 Ap Oy (13-2) 
1 


In this way, we can evaluate the discrimination which cannot be exceeded, given the wave- 
form, the modulation method, and the noise intensity. By this means, it is clear that we 
can also determine in many cases how close the given receiver is to being ideal in the 
presence of strong noise, In fact, if it turns out that the probability determined experi- 
mentally by the "articulation" experiment described above is close to the probability given 
by Eq. (13-1), this means that for the given kind of transmission, the given receiver pro- 
vides almost the maximum protection against strong fluctuation noise. This also means 
that other receivers cannot provide more protection against this noise, when the waveforms 
F(t) and F(t) are transmitted. Clearly, the value of the method described can be 
determined only after applying it in practice. 

In the method studied here, we use waveforms which can take on two discrete values. 


Of course, one can also develop a method of evaluation which uses many discrete waveforms, 


13-3 Maximum discrimination for phase modulation 


To illustrate the method discussed in the preceding section, we apply it to the special 
case ofphase modulation. In order to test the influence of noise, we transmit either the 


waveform 


F(t) = sin (Lt, for -@/2@<t< %/2 
(13-3) 
F(t) = 0, for t<-@/2 and t> Ul 
or the absence of any waveform, i.e. 
Fa(t) = 0 (13-4) 


Suppose we study phase modulation, for which the transmitted signal equals 
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Ap(t) = U, cos [wt + mF(t)] (13-5) 


Then, in our case, we obtain 


es) = U,cos[wit+msin(Lt] for -%/2<t< y/2 (13-6) 
ar = Uy cos wt for t< - © /2 and t >t, /2 
(13-7) 
and 
sy = U, cos wt (13-8) 


Substituting these expressions into (13-2), and assuming for simplicity that w >> (L 
° 
and that BO ey nm is an integer, we obtain 


a” = (07/67) (1 - I,(m)) (13-9) 


where J ,(m) is the Bessel function of order zero with argument m, and 


Un v (13-10) 


Substituting this value of a into Eq. (13-1), we obtain an upper bound for the probability of 
correct discrimination of the waveforms F,(t) and F(t) at the receiver output, in the 
presence of noise of intensity oO. 

In Figure 13-1, the quantity m is plotted as abscissa, and curve 1 gives 1 - J (m) 
as the ordinate. The latter expression completely determines the quantity V(«) Sppeexive 
in Eq. (13-1), if we specify the value of Q/go-. Therefore, we can also plot the value of 
V(a@) along the ordinate axis in the figure, if we specify Q/o@; this has been done for the 
values Q/o = 1, 2, 3, 4, 6. As the figure shows, the quantity V(a) increases when m>4, 
Clearly, the reason for this is the following: For such large values of m, in order for 
there to be an error in interpreting the waveform F(t), at the time when this waveform is 
expected, the noise waveform must take on a value so large that phase modulation no longer 
provides good protection against the noise, For sucha large noise waveform, it is clear 
that the transmitted waveforms cannot be properly distinguished in the midst of the noise 


at the receiver output, regardless of the value of the modulation index m. 


13-4 Maximum discrimination for weak noise 


To clarify the special features which strong noise introduces, we now determine the 


maximum discrimination, starting from the theory derived in previous chapters for the 
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case of weak noise. When the waveform F(t) is transmitted in the presence of weak noise, 


then at the output of the ideal receiver we obtain the waveform 


F(t) + W*(t) (13-11) 


and when F(t) is transmitted, we obtain the waveform 


F(t) + W(t) (13-12) 


where W*(t) is the normal fluctuation noise with the intensity o* given by Eq. (9-27). We 
confine ourselves to the case where g-* does not depend on the frequency, as is always the 
case except for integral modulation. As shown in Section 4-1, the probability that we 
correctly determine which of the waveforms F(t) or F(t) was transmitted, using the 


function (13-11) and (13-12) and the ideal indicator (ideal ear), is equal to 


]- Po = 1] - V( a) (13-13) 
where 
npids ae Ba ee SE 
ee [Fj (t) - Fy (t) J 
ee Se (13-14) 
2o0°* 


Obviously, according to what was said in Section 13-2, the size of (13-13) cannot be greater 
than the probability given by Eq. (13-1). If this does not turn out to be the case, it means 
that the probability (13-13) was improperly calculated, which means that the assumption 

that the noise is sufficiently weak is not valid. In the next section, we compare this method 
of evaluating the maximum discrimination with the general method discussed in Section 13-2, 


using phase modulation as an example, 


13-5 Maximum discrimination for weak noise and phase modulation 


We now apply what was said in the preceding section to the case of phase modulation, 
which we studied in Section 13-3, Assuming that the test waveforms F(t) and F(t) are 


defined as before by Eqs. (13-3) and (13-4), and using Eqs. (13-14) and (10-11), we obtain 


> 2 +%_/2 


U"-m re 
2 fe) J shee Q™m 
a —— sin-tLt dt = 
fon = eee ay 
46° 46 


G12 


For simplicity, we took Bites / a equal to an integer, and denctee Tb, us [2 by os In 


Figure 13-1, curve 2 gives the dependence of the quantity m 214 on m, Thus, using this 
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curve and giving different values to the ratio Q/o-, we can determine the quantity V(a*), 
just as in Section 13-3 we determined the quantity V(a), using curve 1. As we see from the 
figure, the two curves are close together only when the modulation index m <2. In the case 
where m> 2, the value of V(a) given by curve 1 is much larger than the value V(a *) 
given by curve 2, It follows from this that the quantity (13-13), determined according to the 
formula derived for weak noise, is incorrect for m > 2, which means that the weak noise 
theory is not applicable in this case. Clearly, this result can be interpreted as follows: 

As long as the test waveform F(t) gives a small modulation index m <2, it ismasked at 
the receiver output by noise waveforms which are small enough so that Eqs. (13-11) and 
(13-12) are valid. In the case where the test waveform produces a modulation index m> 2, 
it is masked at the receiver output only when the noise waveform is so large at the time when 


F(t) is transmitted that Eq. (13-13) and the weak noise theory are not valid. 
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APPENDICES 


Appendix A, The specific energy of high-frequency waveforms 
As is well-known, a high-frequency signal can be represented quite generally as 
A(t) = UL (t) cos [at + O(t)] (A-1) 


The specific energy of this signal is 


Q% = T AM) = T US (t) cos” ttt Hit) 


Now if we assume, as is usually the case, that w is so large that the frequencies which 
effectively matter in the eSprce on cos|{ 2 a) t + 20(t) are all higher than the frequen- 
cies contained in the function ue mt)» and that ae constant component of cos { 2a t + 26(t)] 
can be set equal to zero for the same reason, then by Eq. (2-26), we obtain 
+T/ 2 
2 1 2 
T uf) = 5 j Utd) dt (A-2) 
-T/2 


Appendix B, Representation of normal fluctuation noise by two amplitude-modulated waves 


We consider normal fluctuation noise with frequencies from 4 /T to YV/T and con- 


stant intensity, and write 


Vv = 
p22 oa a (B-1) 


Let L, and n be integers. Then the waveform (2-54) can be written as 


= 
a 
= 
iy 


n 
won ake 92g +2i-1 sin — ~ (b, +i)t oo. 424 C8 SEL on] 


it 


~ ls eee 
Sere Se 2p eter | Oe eee 


77 ae . 2x 
ht + cos qr it sin =F s.'] 
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Setting (2/T) L, = Ww and factoring out cos wet and sin wot, we obtain 


W (t) = cs y (9 sin 2x it +0 cos one it) cos w _t 
Au vu VT eon 28 +2i-1 T 2h +21 T fe) 
or os an. ae 
+ 6 ; cos it - 0 . sin it) sin wt (B-3) 
2 2 T 
VT pte 2h, +21 i T 286+ i fo) 


Finally, adding terms with the same absolute value of i, we obtain 


n 
o . on. 
W (t) = c ’ - 6 see: ) sin — it 
fev Ba ae 2h ,+2i-1 2h, 2i-1 T 


an. 
+ (82g +21 + ya er cos—a- | cos wt 


it 


2x 
[2g zie TDP Seay ee = 


ee a ear 2 
+ 28. 221 - 92.0, +23) sin - it sin wot (B-4) 
Here we have neglected the terms with i = 0, whichis permissible if we take T large 
enough, since if T is increased while the frequencies .w/T and W/T are kept the 
same, the number of terms will increase, while each term becomes arbitrarily small. We 
now introduce the notation 


= V2 6 


6 | . . 
2H,+2i-1 2p-2i-1 2i-1 


P29 +2i-1 + 924 -2i-1 = ¥2 05, 


(B-5) 
= W 
oy eg eer a ey 
fo) (eo) 
. : ' 
ong sav” ny aa 
3) fo) 
where, according to Section 2-5, the @'. eu @)., 04. are (mutually) independent 


2i-1? “2i-1’ “2i’? ~2i 
random variables. Substituting these quantities in (B-4), we obtain 
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Wa wl) = Wgo-7 2%" = f2 Watt) sin @).t + 2 Wieck) cosw t (B-6) 


where a 
Oo 
WwW! (t) = (@:. sin it + @!. cos it) 
9 YT ne 2i-1 T 21 T 
om = 2x 2x 
Ww = u og 3 " - 
1 ntt) eT ve (85.4 sin =p it + 855 cos —- it) 

: ; (VY - 2) 
are independent normal fluctuation processes with frequencies from zero to ne 
The quantity @ = ems, Lo = si ise Vo+ Le ) is the mean angular frequency of the 

° T Me, 2° Tae 6 me ge 
rocess W t 
. Yu»! 


Appendix C. The instantaneous value of normal fluctuation noise 


We now find the value of normal fluctuation noise with constant intensity at some 


instant of time t = the According to Eqs. (2-54) and (2-74), we have 


an 
T 


i 


» 
oO 
— 8 
fT: pew W2g-1 


uv 
= Ngee — gt, eee" ae £t,) 8, 


JT 
- + 1 
PVs. (C-1) 


where 8, is a normal random variable. Introducing fy = V/T and ya = A/T, the limits 


- an 
WwW (t)) ers Lt, + 859 cos Lt) 


x» 


i 


u 


of the frequency band of the process under consideration, we find that for large T 
1) =OVJEL, - C-2 
Wy wf) oA) - te 8) (C-2) 


The rms value of Mi ylty) is oi) - a ,» which agrees with (2-57). 
) 
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Appendix D. Normal fluctuation noise made up of arbitrary pulses 


We consider the passage of normal fluctuation noise through a linear system, Let the 
process Wy p(t) given by Eqs. (2-54) and (2-27), and consisting of the very short pulses 
> 


(2-28), act upon the input of the system. This process can be written as 


U 
o - 2x 2m 
Wahl = oh = (859-1 sin = Qt+ 952 cos =r Lt) 


where y can be arbitrarily large, if the pulses are taken to be short enough. The process 


at the output of the system is 


v 
SE ocak ok(L/T) _ [2x 2n 
W*(t) ot ae 954-1 vin | SE ge + 066] + 830 cos [Sf ae + 006] 


(D-1) 
where k(Q/T) exp {j@(2/T) ] is the complex transfer coefficient of the system at the 


frequency £/T,. Expanding the sine and cosine terms in this expression, we obtain 


S 
w(t) = es sei) (0591 cos O/T) - 09 sin WL/T)] sin SE gt 


+ [6,9 1 sin 6(4/T) + 958 cos $§(£/T) ] cos ae bt (D-2) 


According to Eqs. (2-74) and (2-75), we have 


il 


V cos” 9(2/1) + sin® (0/7) OF) 1 = OS 


859-1 C08 B(L/T) - @, 4 sin B(L/T) 


and 


959-1 sin 0(2/T) + ry) cos $(2/T) = NJ ain’ O(2L/T) + ease 6(0/T) Ox, = ew) 


where OF g-1 and 83 9 are independent normal random variables, since the condition (2-76) 


is satisfied, i.e. , 
cos O(£/T) sin O(2/T) - sin O(2/T) cos O(2/T) =0 


Accordingly, we obtain 
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y 
W(t) = Ps uit) (859) sin gt + 0% y cos oe Ot) (D-3) 
where 
o*(L/T) = ok(L/T) (Dat) 


As is evident from this expression, the phase characteristic $(£/T) of the system does not 
affect the statistical properties of the process (D-3). We shall call the process W*(t) 
normal fluctuation noise with the variable intensity o*(0/T) = o(f). 

The process Wa,y'')> acting on the input of the system, consists of short pulses, Each 
of these pulses produces a pulse at the output of the system, with a form determined by the 
complex transfer coefficient k(£/T) exp [ j(#/T)]. Thus, we can regard the process 
W*(t) as being formed by the superposition of a large number of similar pulses, which are 
randomly distributed in time. Moreover, the intensity of the process at the output of the 
system can be found directly from the spectral function of the output pulses. In fact, the 


modulus of the spectral function of the k-th output pulse is 


|e. (2x £/T)| = a, k(B/T) (D-5) 


where q. is defined by Eq. (2-32), and is the modulus of the k-th input pulse, since the 


input pulses are infinitely narrow. : Therefore, in view of (2-39) and (D-4), we obtain 


n n 
OBIT) = ok b/T) = % Et K(/T) = 2 Xe bee] 


Thus, the sum of a large number of pulses which are randomly distributed in time and which 


have (D-5) as the modulus of their spectral function, is the normal fluctuation noise (D-3) 


~/2 iS 2 
o*(f) = = es |g, (2s£) | (D-6) 


where the sum is over all pulses in the interval -T/2, +T/2. 


with intensity 


It is not hard to show, by considerations similar to those given above, that the sum 


W'(t) + W(t) + WEM(t) 4 oe (D-7) 


1. See, e.g., V. A. Kotel'nikov and A. M. Nikolayev, "Elements of Radio Engineering", 
Part I, Svyaz'tekhizdat (1950), Section 8-5, 
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of several fluctuation noises with variable intensities is also a normal fluctuation noise, 


with intensity given by 
my nae if ne mm 
o*(f) = o'(f) +o" (f) + OM(f) +--- (D-8) 


where o'(f), o''(f), o(f), ... are the intensities of the noises W*'(t), W*''(t), W*'"(t) 


» eee 


Thus, a process which consists of randomly distributed pulses with different shapes is also 
a normal fluctuation noise. 


We now find the effective value of the normal fluctuation noise (D-3) with variable 


intensity. According to the theory of Fourier series, the square of the effective value is 


ee a v o*-(¢ ) 
we(y ey ar (0591 +) 


where f9 roam Averaging this value over an ensemble of realizations, we obtain 


Maes, 
¥e = 2 
E W*"(t) oe o* (fy) (fp 1 > fp) 


: 12 2 
since E 054-1 = E O59 =1, and 041 - 7) = 1/T. As T increases, thedifference 
f+ - fp goes to zero, and 


E w(t) | o*7(£) df 
V—> o ° 
T—> © 


whence, for sufficiently large T, the effective value of the process (D-3) is 


4 E w(t) = vf o*-(f) af (D-9) 
(e) 


If o*(f) is zero from ivi on, we Say that fui is the upper limit of the frequency band. 
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This investigation of the performance of various communications systems in the 
presence of additive Gaussian noise is a basic work in information theory. In it, Dr. 
Kotel’nikov, one of the few electronics engineers to be elected an Academician of 
the Academy of Sciences of the U.S.S.R., provides remarkably clear expositions of 
his important contributions, many of which anticipated similar developments in the 
West by a considerable period of time. 


To enable the reader to use the book more readily, the author first sets the frame- 
work of the problems to be considered and helps the reader to sharpen his mathe- 
matical tools for the rest of the book. The second part then treats of the transmission 


of discrete messages, while the third and fourth parts cover transmission of a 


contiluous parameter value and continuous time functions. Each of these parts 
contains examples of the application of general theory to special types of com- 
munications systems: AM, FM, PAM, PPM, SSB, and others. 


The author's statistical analysis of his problems invokes what are now called 
decision theory techniques. His approach, however, is that of the practical engineer, 
necessitating only elementary notions of probability theory. The reader with some 
familiarity with Fourier series, discrete and continuous probabilities and probability 
densities will have little trouble in following the text. Extensive use-of geometrical 
models of the signalling and detection process is very helpful to the student. 
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